XMLHttpRequest, callbacks and promises
While following a React tutorial, I had to grab some data from an external API using the fetch API. The logic seemed easy to understand and the data were rendered correctly. Nevertheless, I couldn’t fully grasp the concept of “returning a promise” and I wanted to understand a bit more in detail how the fetch API worked under the hood. Reading about fetch led me to read about promises and eventually about XMLHttpRequest and callbacks. I decided to slow down along my learning path to try to understand how network requests had developed and gotten better throughout the years. After all, we can live a better present when we are aware of the past :-). That is how I have ended up writing this post. I am planning on a next one in the coming weeks about the fetch API itself and async/await. Please, if you spot any outrageous mistakes in my article leave a comment as I am open to constructive feedback.
Javascript — an asynchronous language
Javascript is a single-threaded language and normally works synchronously. This means that, when the code starts running, the tasks are performed one after another in sequence and, to move forward to a new code block, the previous one needs to have been tackled completely. A common issue arising from the synchronous nature of Javascript is that the code is subjected to get “blocked” whenever an action requires some extra time to be performed. An example of this is when we need to grab some data from an external API. The entire code would slow down if we had to wait for the data to be fully retrieved. This is the reason why the concept of asynchronous Javascript is important.
Async, simply put it, means to start something now and finish it later.
What has this to do with XMLHttpRequest you might be wondering? XMLHttpRequest supports both synchronous and asynchronous communications.
Let’s see how grabbing some data from an external api would work with an asynchronous function:
- The function retrieving the data starts running.
- The code running inside this function is “handed over” to a second thread and in the meantime, the rest of the Javascript code keeps on running.
- When the data are 100% retrieved (and in the meantime our code kept on running), one or more callback functions are fired to perform actions with the data.
The above sounds like a lot to digest. At least it did to me as I have been wondering:
- How is the data retrieved?
- How does the concept of callback function come into the picture?
Let’s try to reply to these questions below.
The XMLHttpRequest
Let’s have a look, by keeping it very simple, at how XMLHttpRequest works:
- We create the XMLHttpRequest object and store it in a variable so that is easy to reuse. This object comes with a lot of useful methods to retrieve the data.
let request = new XMLHttpRequest();2. We use the open method to set up the request. This takes three arguments: the type of request, the endpoint (basically a URL where to grab the data from), and a boolean value which indicates whether the request is asynchronous or synchronous. This third argument is optional. If its value is “true” or left empty, then the request will be asynchronous. If its value is “false” then the request will be synchronous. For this example I’ll be using the JSON placeholder API:
request.open(“GET”, “https://jsonplaceholder.typicode.com/todos/");3. We send the request:
request.send();If you are reading this, I recommend to paste the code below in your text editor and then open the console. You’ll se the actual request object (as we are console.log it) and it will make more sense.
let request = new XMLHttpRequest();
request.open("GET", "https://jsonplaceholder.typicode.com/todos/");
request.send();
console.log(request);Among the properties, you’ll find the ReadyState property. Its value can be a number between 0 and 4 and it is super important as it tells us at which stage our request is. Let’s see what these numbers mean:
0. The request has been initialized yet (before we call the open method)
- The request has been initialized (after we use the open method)
- The request has been sent
- The request is being processed
- The request is complete and we finally got the data back (or maybe an error)
By using the ReadyState property we can track our request and “get notified” when the data are retrieved (when the value will be 4) so that we can perform actions with them. We can do this by adding the event listener readystatechange to the request, which will trigger a callback function whenever a change of status is detected in our request.
Let’s make our code a bit more advanced by implementing what is described above:
const request = new XMLHttpRequest();
request.addEventListener("readystatechange", () => {
console.log(request.readystate);
if (request.readyState === 4 && request.status === 200) {
console.log(request.responseText);
} else if (request.readyState === 4) {
console.log("could not fetch the data");
}
});
request.open("GET", "https://jsonplaceholder.typicode.com/todos/");
request.send();In the code, we console.log the data (request.responseText) when the readystate is 4 and the status 200. In case there is an error in retrieving the data then the message “could not fetch the data” is displayed. The reason why we need to specify a status is that in case the URL endpoint would be spelled with a mistake, the request would still try to go through all 4 stages anyway and the responseText would be empty.
Normally we would wrap the request function inside a variable so to make it more reusable. To do this, below we use the arrow function notation:
const getData = () => {
const request = new XMLHttpRequest();
request.addEventListener("readystatechange", () => {
console.log(request, request.readystate);
if (request.readyState === 4 && request.status === 200) {
console.log(request.responseText);
} else if (request.readyState === 4) {
console.log("could not fetch the data");
}
});
request.open("GET", "https://jsonplaceholder.typicode.com/todos/");
request.send();
};Now we can call this function whenever we want by writing:
getData()This is where things got a bit more complicated for me. At this point, many articles/tutorials started talking about callback functions and how we can add them to our request function to perform actions with our data.
A callback function can be defined as an argument of getData using the arrow function notation as shown below:
getData(()=>{
...here is where we define the callback function...
})To use it in our original request function though, we need to pass it there as a parameter (note that the name “callback” is arbitrary):
const getData = (callback)=>{
....our code...
}As mentioned at the beginning of this post, the callback function is fired when the data are retrieved. In our getData function above, we console.log either the data (request.responseText) or an error message. Normally, instead of console.log we would call the callback function to perform some actions with our data.
Let’s see how we can do this:
- First, we define what the callback function does. In this case it takes two parameters: “err” and “data” (it is a convention to mention the error first). We’ll see below more in detail what this code does.
getData((err, data) => {
if (err) {
console.log(err);
} else {
console.log(data);
}
});2. Then we call the callback function inside the getData function declaration as follows:
const getData = (callback) => {
const request = new XMLHttpRequest();
request.addEventListener("readystatechange", () => {
if (request.readyState === 4 && request.status === 200) {
let data = JSON.parse(request.responseText);
callback(undefined, data);
} else if (request.readyState === 4) {
callback("could not fetch data", undefined);
}
});
request.open("GET", "https://jsonplaceholder.typicode.com/todos/");
request.send();
};The callback function can take different arguments and, according to their value, our function will perform different actions: in this case either display an error or the data. In the example above, the first time we call the callback the arguments are: undefined, data. The second time: “could not fetch data”, undefined.
According to the callback function declaration, if an error is found, which in the example above it is the string “could not fetch data”, then we display the error message in the console (note also that the data is marked as undefined). If there is no error (the argument for the err is undefined) then we console.log the data.
Note that in the code above we have also introduced the variable “data” which is the Javascript object we get by parsing the JSON string (I am not entering into details about JSON as I think it could be a topic for another post).
In this example, the callback function performs a simple task by either displaying the data or an error message. I believe though that understanding its mechanism gives an idea of how powerful the concept is. We can call the getData function whenever we want in our code and every time we can define a different callback function to perform different actions with the data.
We should also note that our getData function behaves asynchronously and we can prove this by writing the following code:
console.log("1");
console.log("2");
getData((err, data) => {
if (err) {
console.log(err);
} else {
console.log(data);
}
});
console.log("3");When we check the console we will see 1, 2, 3, and then the data. Our code works exactly as expected: it console.log “1”, then “2”, then while the data are being retrieved it console.log “3” and only after the callback function is fired.
Callback Hell
There might be cases where we need to get several files one after the other and we have to wait for the first batch of data to be fetched before moving on to the second. This could be the case when we grab data from different APIs and we might use the first batch of data to make a request to a second API.
We can simulate this:
Imagine that inside our working folder we have another folder containing three different JSON files we want to fetch one after the other.
We can modify the code as follows:
const getData = (resource, callback) => {
const request = new XMLHttpRequest();
request.addEventListener("readystatechange", () => {
if (request.readyState === 4 && request.status === 200) {
let data = JSON.parse(request.responseText);
callback(undefined, data);
} else if (request.readyState === 4) {
callback("could not fetch data", undefined);
}
});
request.open("GET", resource);
request.send();
};
getData("./data/list.json", (err, data) => {
console.log(data);
});Here we add the “resource” parameter and we set it to “./data/list2.json” to begin with. In the console, we will see the content of the list.json file.
If we wanted to retrieve all the files one after the other we would write the code as follows:
getData("./data/list.json", (err, data) => {
console.log(data);
getData("./data/list2.json", (err, data) => {
console.log(data);
getData("./data/list3.json", (err, data) => {
console.log(data);
});
});
});The code above is an example of callback hell, nesting callbacks inside other callbacks which eventually becomes very difficult to maintain. A way to avoid this messy code is to use what are called promises.
The anatomy of a promise
I like to think of a promise as an empty container that will be filled with data in the future. This container is where all the asynchronous code to retrieve the data resides:
const promise = new Promise((resolve, reject) => {
//asynchronous code goes here
});The promise takes a callback function which takes two arguments, resolve and reject, which are both functions. If the data comes back successfully then the resolve() function is called, if there is an error in retrieving the data then reject() is fired.
When we create a promise we then have to be ready for two possible outcomes: error (promise rejected), success (the promise is resolved).
How do we know when the request has been processed and the data are possibly available? For this we use the .then method which takes two arguments:
- a callback which is fired when the promise is resolved, taking as argument the actual data (the first in the example below).
- a callback which is fired when the promise is rejected and whose argument is whatever you pass to the reject function (the second in the example below figuring the err as argument).
This is how the function would look:
promise.then((data) => {console.log(data);
},(err)=>{
console.log(err)
});
Alternatively, instead of having a second callback function, we can add the .catch method which fires the callback function only if the promise is rejected. Normally this syntax is considered cleaner:
getSomething().then((data)=>{
console.log(data)
}).catch((err)=>{
console.log(err)
})With the above knowledge we can re-write our getData function as follows:
const getData = (resource) => {
return new Promise((resolve, reject) => {
let request = new XMLHttpRequest();
request.addEventListener("readystatechange", () => {
if (request.readyState === 4 && request.status === 200) {
let data = JSON.parse(request.responseText);
resolve(data);
} else if (request.readyState === 4) {
reject("error getting resources");
}
});
request.open("GET", resource);
request.send();
});
};
getData("./data/list.json")
.then((data) => {
console.log("promise resolved", data);
})
.catch((err) => {
console.log("promise rejected", err);
});
getData doesn’t take a callback function anymore as this has been replaced by the promise which we tackle with the .then method.
If the promise is resolved, the message “promise resolved” and the actual data are printed to the console. If the promise is rejected the message will be “promise rejected, error getting resources".
Essentially, a promise is a returned object to which you attach callbacks, instead of passing callbacks into a function. (source MDN)
In the above example, we are retrieving the data only from one file. How could we grab the data from all files in sequential order? The last chapter will try to explain this by introducing the concept of promise chaining.
Chaining promises
Sometimes you might need some data to be fully retrieved to go out and fetch another bundle of data. For tasks like this, we can chain multiple promises.
If we wanted to retrieve the data sequentially we would write the code as follows:
getData("./data/list.json")
.then((data) => {
console.log("promise 1 resolved", data);
return getData("./data/list2.json");
})
.then((data) => {
console.log("promise 2 resolved", data);
return getData("./data/list3.json");
})
.then((data) => {
console.log("promise 3 resolved", data);
})
.catch((err) => {
console.log("promise rejected", err);
});By calling getData() the first time we create a promise to get the data from ./data/list.json. Only when this promise is fulfilled (either rejected or resolved) we run a callback function:
- if the data are back successfully we create another promise by writing: return getData(“./data/list2.json”), this creates a new promise which we can tackle with another .then method.
- if there is an error then the catch method is fired.
It is important to return the promise for the callbacks to catch the result from the previous promise.
Moving forward, we either get the second batch of data (“./data/list2.json”), which means that the successful callback will be fired and another promise will be made to grab “./data/list3.json” or, in case of an error, the catch method will be fired. This process continues until all the data are retrieved sequentially. With this structure in place, creating a promise inside another promise, we make sure that all the data are retrieved before starting to grab new data. And, whenever there is an error retrieving something, the code stops running.
I hope this has helped you a bit in understanding the complex topic of network requests, synchronous and asynchronous. Next, I’ll write about the fetch API.
