Blockchain Direct HTTP Requests via the Internet Computer (2024)

While there’s plenty of optimism in Web3 about a future where everything happens on a blockchain, we’re a long way from that being a reality. The vast majority of useful data is generated in traditional data centers or cloud computing infrastructure and interfaced using familiar tools that largely leverage HTTP or HTTPS.

Decentralized applications running on any of the Layer 1 blockchains, like Ethereum or Solana, interface with HTTP-based services using what’s known as an oracle. These oracles act as trusted middleware to allow for the creation of a hybrid smart contact, where on-chain code can interact with off-chain infrastructure and data. Chainlink Network, a popular decentralized oracle network, offers a short explainer video on what this looks like.

The Trouble with Oracles

While oracles are the primary means of connecting dApps (decentralized apps) with off-chain data and infrastructure, there are some disadvantages. The requests are indirect, meaning you aren’t making an API call directly to the source of the data you want to query — the oracle does this for you and then your dApp needs to trust the response the oracle comes back with. This approach also comes with fees associated with using the oracle as a third-party intermediary.

DFINITY Foundation, one of the largest contributors to the Internet Computer, a layer 1 blockchain, is proposing an alternative approach where dApps can make HTTP requests directly using an API integrated into the blockchain.

In an interview with The New Stack, Dieter Sommer, Technical Program Manager for the DFINITY Foundation, explains the challenge of relying on oracles this way. “Everybody who wants to do anything reasonable needs some form of integrating with Web 2, and all other blockchains use oracles for this,” he said. “Oracles are outside services, so if you rely on an oracle to connect to Web 2, the oracle does all the HTTP stuff. This also means you introduce lots of new trust assumptions; for example in the standard model of using the Chainlink oracle, you make a call to one oracle provider and this provider needs to be trusted by you, which is a very weak model.”

An API to Make HTTP Calls Directly

DFINITY Foundation uses some slightly different terminology to explain how the Internet Computer blockchain infrastructure works. With a foundation of The Internet Computer Protocol, the Internet Computer hosts smart contracts called Canisters that are a combination of WebAssembly bytecode and memory pages where this code runs. Deploying a Canister means that the corresponding code and state gets replicated to all nodes on the subnet it is deployed on.

This concept of replication is one of the reasons most blockchains are using oracles to make HTTP requests today. In the design of the Internet Computer currently, each replica would make the same HTTP call to an external service. But the HTTP response returned to each replica might be different, because there could be variation in timestamps or IDs. When the replicas all get a slightly different response, it’s impossible to achieve consensus — which effectively breaks the subnet.

In the upcoming Chromium release of the Internet Computer, there’s a new approach to solving this problem and providing the blockchain with a direct integration using an API to make HTTP calls. This removes the trust assumptions required to use an oracle and theoretically simplifies the process of accessing off-chain data.

With an asynchronous API provided via the management canister, every node will make the same HTTP request. As each node receives a response, they sign the response and gossip it to the other nodes. Once the consensus layer aggregates enough signatures, it will include the response in the blockchain. When the block is finalized, the response is passed back to the execution layer, which in turn resumes the computation that originated the HTTP request.

Navigating Inconsistent Responses

When all nodes receive the same response at approximately the same time, this approach works flawlessly. This should work even in scenarios where a malicious node reports false information, as long as enough nodes come back with the same response.

As Sommer puts it, “All the nodes in a subnet make the request and only if consensus is successful, meaning at least two-thirds of the replicas agree on the result, then it is replied back to the canister as a result. This allows for a secure call outside, without any external third parties to rely on. Our consensus protocol is flexible enough to allow for such extension.”

A more complicated scenario is when the requests are semantically the same, but potentially have minor differences that don’t matter for the computational result. Instead of failing to reach a consensus, you can code around these inconsistencies using a function that transforms the response by only surfacing the portion of the response that is required for the computation. Take an example like needing to return a string of text where the text is packaged in a response with a timestamp. If the string of text is the same in all cases, it doesn’t matter that the timestamps vary and you can use the function to throw those out.

For the initial release, only GET requests are supported. The longer-term plan is also to support POST requests. In watching the DFINITY Foundation video covering this new feature in additional details, Ivan Malison, a software engineer at DFINITY, explains that POST requests are more complicated. He showed an example of a credit card payment. You wouldn’t want to attempt to charge the same card multiple times or get a different response to your POST request like a success message one time and a decline the next. The video provided Stripe’s Idempotency for safe API retries as an example of how to implement this correctly in the future.