XHR, Part 1

XHR and AJAX

• AJAX stands for Asynchronous JavaScript and XML, and is a term that describes an approach to requesting server-side data and pushing data to the server.
• AJAX relies upon JavaScript, the DOM, and the XHR object (AKA the XMLHttpRequest object).
• The mention of XML in both AJAX and XMLHttpRequest is misleading; it is only one of many different data formats that can be used. The other formats include:
  • HTML
  • CSV (comma-separated values)
  • JSON (JavaScript Object Notation)
  • Other text strings with custom delimiters (other than commas)
• This approach enables you to receive data from, and pass data to, the server without needing user input.
  • This asynchronous setup breaks from the traditional Web approach of synchronous communication, in which the user takes an action and waits for the server response.
  • One example is a chat application.
    • There could be XHR calls every couple of seconds that take in whatever you are typing and updates your chat window with what the other people have typed.
    • Rather than typing and then clicking 'Send', the update happens automatically.
    • How frequently you update, or poll for new data, is a very important consideration and will depend on the application as well as the available bandwidth.
    • setInterval() would be used to trigger the periodic updates.
  • The chat application example also showcases another important detail in these approaches: updates are made to just one part of the screen rather than forcing an entire page reload.
• The degree to which AJAX is implemented falls into a range from completely relying on it, to just using it for some enhancements.
  • The "full approach" will use a bare-bones HTML template, with areas in the layout where the data is inserted.
  • For those applications, disabling JavaScript means that they cannot be used, unless a <noscript></noscript> alternative has been created. In such applications the concepts of graceful degradation and progressive enhancement are harder to implement.
  • If the usage is for a minor enhancement (such as a news ticker) then it is much easier to support graceful degradation and progressive enhancement.
  • Spiders will vary in how much they can index, if your web pages are empty shells depending on XHR calls to populate their content. Google's spiders will do just fine in pulling in the data; others are less capable. It also depends on whether the spider is running the appropriate interpreter.
• Before the rise of AJAX, an <iframe></iframe> was used to make asynchronous calls for data.
  • The <iframe></iframe> would have its visibility set to hidden and its height/width set to 1px. Hiding visibility is preferable to display: none because the latter approach resulted in some browsers not allowing access to the data in the iframe.
  • Since the iframe has a src attribute that can be dynamically modified (via JavaScript), it allows other documents to be loaded there and content to be written to it as well (it has its own window and document object).
  • Thus the iframe could contain a form that you would update, and then submit as desired.
  • Essentially, you end up with an embedded data storage and retrieval window.
  • It's possible you will still see this implementation, especially in old web applications written many years ago.

XHR Pitfalls

While asynchronous calls have a lot to offer, there are also numerous considerations and drawbacks that come with these implementations:

• Overtaxing the Browser
  • The shift to XHR has put a lot of pressure on the browser, as processing has been shifted off the server and moved to the client.
  • There are limits to this rich client model, including memory, CPU, and available data streams into the browser.
  • This has been exacerbated by tabbed browsing, which allows the user to have open multiple tabs that could all be requesting data asynchonously.
  • There are bandwidth considerations as well (mobile users on slower networks will experience noticeable lags).
• Page Changes Require Feedback
  • When updating one part of the page, it is essential to somehow notify the user that a change has occurred there.
  • One approach is to have the background color change for that area, then fade back to the regular background color.
  • This benefits users with large monitors who can see the change, but it does not help blind users or users with a screen magnifier that is focused on a different part of the screen (they never see the highlight).
  • While you might be tempted to include some code to shift the user's focus to the newly updated area (to "help" the user with the screen reader), you just pulled them away from whatever else they were doing. The focus may shift anyway when the content is updated, regardless of any special code that you wrote.
  • Also make sure that whatever you are adding or updating is marked up properly, so that different ways of accessing document areas in screen readers (such as by header tags and links) can reflect the change.
  • Areas where content is being modified need aria-live="polite" specified, so screen readers get notified of the change. Otherwise they rely on their initial snapshot of the DOM when the page loaded, and they are unlikely to learn of the change.
  • If you are temporarily showing a message, perhaps an indication that saving is occurring and has completed, also specify role="status" for the tag holding the message.
  • You can even specify what changes gets communicated via aria-relevant.
• Data Loading / Saving Requires Notification
  • If data is being pulled from the server or written to the server it is very helpful to display a message indicating that loading / saving is occurring.
  • Otherwise the user has no idea what is happening, because the standard response of a page reloading is not happening.
  • The accessibility issues noted for the previous item apply here.
• Data Security is a Concern
  • Even though the data exchange is asychronous, someone can still be grabbing the packets as they are sent to the server or received from the server.
  • Usage of HTTPS for all traffic is important.
• Breaking Expected Behavior for the Browser's Back Button
  • Users expect the Back button to take them back one step, but in these applications the Back button moves back an entire page, not just one content update.
  • The History API is the best solution, as it allows you to save the page "state" and associate that with changes in the hash value (the text after the # sign in the URL).
• Breaking Expected Behavior for the Bookmark / Favorites Functionality
  • While the URL can still be bookmarked, this will just load the initial page before any XHR-driven modifications.
  • The History API is again the solution to this, but that relies on the user having visited the page so that their browser has the "state" stored. If they wipe their browser cache, or someone else shared with them the link, then they won't get the proper page "state" to appear, unless you have some sort of server-side fallback.
• Reflowing the Page
  • When adding or removing content via XHR, there is always the possibility that the user will be reading something below where the change occurred and will lose their place on the page when content shifts.
  • Such an experience is quite disorienting, so avoid inserting content into a column that would trigger page scrolling.
• Error Reporting
  • Since this is running in the client, any errors that occur there will not be reflected in the server error logs.
  • Unless there is a reporting feedback mechanism put in place, those problems will not be known to you.
• Breaking the Asynchronous Communication
  • Use of alert(), confirm(), and prompt() breaks the communication pattern because all JavaScript processing in the main thread (where XHR is executing) stops until those prompts are acted upon by the user.

Situations Where XHR Can Add Value

• Returning search results as the user types in the query.
• Updating small amounts of information (e.g., news headings, stock ticker, weather report).
• Auto-saving form information periodically.
• Detecting whether something is in stock and reporting that back as soon as the product options are chosen.
• Pulling data from a large dataset as it is needed; examples include pulling discussion board responses as the user navigates down that thread as well as how Google Maps pulls data from areas around where the user is looking, so that it is ready when the user moves the map in that direction.
• Auto-refresh of data in communication applications such as email and chat.
• Checking form data as it is entered (while JavaScript can do this on its own, an XHR call enables you to query the server and see if that username is already taken, for example).
• Auto-completion of values (or suggesting completed values) as the user is typing.

The XMLHttpRequest Object

• This is a native JavaScript object in modern browsers.
• It is not, however, a Core object (Core objects will work regardless of the environment JavaScript is running in; XMLHttpRequest is web-specific because of its reliance on HTTP).

Properties of the XMLHttpRequest Object

Methods of the XMLHttpRequest Object

HTTP Headers and Status Codes

• A basic overview of headers for older versions of HTTP or an in-depth review of the latest headers in HTTP are available.
• To access these for our request, we can always use xhr.getResponseHeader() or xhr.getAllResponseHeaders(), where xhr represents the XMLHttpRequest object.
• Sending data to the server via 'POST' also requires a content type to be specified:

  xhr.setRequestHeader('Content-Type','application/x-www-form-urlencoded');

The (on)readystatechange Event Handler

• The (on)readystatechange event handler is used specifically with XMLHttpRequest objects and fires each time the state of the request changes.
• This event handler should be assigned to the XMLHttpRequest object after we have called the open() method and before we have called the send() method.
• Note that we only care about readyState 4, since the others do not hold much practical value.

Creating the XMLHttpRequest Object

• Whenever you are working with XHR you must be testing on a server (or running a server on your computer at localhost).
• The file:// protocol is not permitted, which is why running the file locally will fail.
• Use the browser's Developer Tools to inspect the requests you are sending and the information being sent back. There will be a tab related to Network activity and an option within there to filter down to just XHR calls (it may be labeled Fetch/XHR) and responses. You may also be able to configure the Console to show this activity.

XHR Request Method / Function:

// pass the url of the data file, 
// the function that will receive data back,
// and whether data is being sent via POST (optional)
sendRequest : function(url, func, postData) {

  // xhr is the XMLHttpRequest object
  const xhr = new XMLHttpRequest();

  // if that does not work out, 
  // it will be null and the method will exit
  if (!xhr) { return; }

  // if the optional postData is present, use "POST"
  const method = (postData) ? "POST" : "GET";

  // open the request and use asynchronous transfer
  xhr.open(method, url, true);

  // if data is being posted, set appropriate Content-Type
  if (postData) {
    xhr.setRequestHeader('Content-Type','application/x-www-form-urlencoded');
  }

  // use DOM0 because the event does not create 
  // an event object in all browsers
  xhr.onreadystatechange = function() {

    // unless the request is Completed, ignore the readystatechange event
    if (xhr.readyState !== 4) { return; }

    // if there is an error in the data transmission, show an alert and exit
    if (xhr.status !== 200 && xhr.status !== 304) {
       alert('HTTP error ' + xhr.status);
       return;
    }

    // pass the XMLHttpRequest object to the indicated function
    func(xhr);

  }

  // if we have already completed the request, 
  // stop the function so as to not send it again
  if (xhr.readyState === 4) { return; }

  // if we use GET data then postData will be null
  xhr.send(postData);

}

Live Search (XML Data)

Structure (live_search.html)

<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
  <title>XHR Live Search Example</title>
  <meta charset="utf-8" />
  <link rel="stylesheet" href="live_search.css" />
</head>
<body>

<form method="post" action="#">
<p>Search for an Article: <input type="text" id="query" size="20" />
<input type="submit" id="submitButton" value="Search" /></p>
</form>

<div id="results" aria-live="polite"></div>

<script src="live_search.js"></script>
</body>
</html>

Presentation (live_search.css)

body, h1 {font: 75%/1.3 verdana, sans-serif;}
h1 {font-size: 120%; font-style: italic; margin-top: 20px;
    border-top: 1px solid #000; padding-top: 20px;}
input {border: 1px solid #000;}
#waiting {font-size: 80%; font-weight: bold;}

Data File (live_search.xml)

A sample of the XML data:

<result>
  <article_title>AJAX Basics</article_title>
  <article_url>http://www.ajaxbasics.com</article_url>
</result>
<result>
  <article_title>AJAX 123</article_title>
  <article_url>http://www.ajax123.com</article_url>
</result>

View the entire XML data file for the live search example

Behavior (live_search.js)

"use strict";

// create the 'ls' (live search) object
const ls = {

  // reference to the text input box
  searchTextBox : document.querySelector('#query'),

  // reference to the results div
  resultsArea : document.querySelector('#results'),

  // div that holds the waiting message
  waitingHolder : document.createElement('div'),

  // reference to the form element node
  theForm : document.querySelector('form'),

  // initially there is no waiting message
  waitMsg : false,

  // initially there is no timer set
  timer : null,

  // this will hold the XMLHttpRequest data
  searchData : null,

  // flag for tracking if results were found
  resultsFound : false,

  init : function() {

    // disable the submit button; will appear if JS is disabled
    document.querySelector('#submitButton').style.display = 'none';

    // send the request for the XML data
    // data will be sent to the holdSearchResults() method
    // attach a random number to the end to break file caching
    this.sendRequest('live_search.xml?' + Math.random(), ls.holdSearchResults);

    // on every key stroke call the setTimer function
    this.searchTextBox.addEventListener('keyup', this.setTimer, false);

  },

  // store the XML data in the searchData property of this object
  holdSearchResults : function(xhr) {

    ls.searchData = xhr.responseXML;

  },

  setTimer : function() {

   // if the clock is ticking, clear the clock
   if (ls.timer) { clearTimeout(ls.timer); }

   // if there is not currently a waiting message, display one
   if (!ls.waitMsg) {

      ls.waitingHolder.id = 'waiting';
      const theMsg = 'Waiting for you to finish entering your search...';
      const waitingTxt = document.createTextNode(theMsg);
      ls.theForm.appendChild(ls.waitingHolder).appendChild(waitingTxt);
      ls.waitMsg = true;

   }

   // after 1.5 seconds, call the displayResults() function
   // this countdown will be reset if the user keeps typing
   ls.timer = setTimeout(ls.displayResults, 1500);

  },

  displayResults : function() {

    // shorthands
    const data = ls.searchData;
    const results = ls.resultsArea;
    const box = ls.searchTextBox;

    results.innerHTML = '';

    // if the user has deleted all the text in the query, 
    // do not go any further
    // the keypresses involved in deleting the query 
    // will trigger the function
    if (!box.value) { return; }

    // total number of possible results for subsequent loops
    const totalResults = data.getElementsByTagName('result').length;

    // lowercase the query to eliminate mismatches due to case
    const queryValue = box.value.toLowerCase();

    // wipe out the waiting message and flip the waitMsg flag back to false
    ls.waitingHolder.removeChild(ls.waitingHolder.firstChild);
    ls.theForm.removeChild(ls.waitingHolder);
    ls.waitMsg = false;

    // create and append the Search Results header
    const searchHeader = document.createElement('h1');
    const searchHdrTxt = document.createTextNode('Search Results');
    results.appendChild(searchHeader).appendChild(searchHdrTxt);

    // no results found yet
    ls.resultsFound = false;
    
    for (let i=0; i<totalResults; i++) {

      // pull out each article title and lowercase it
      const article = data.getElementsByTagName('article_title')[i].firstChild.nodeValue.toLowerCase();

      // if the query matches the article title
      // then continue and construct the result for that item
      if (article.match(queryValue)) {
          
        ls.resultsFound = true;
        const newLink = document.createElement('a');
        newLink.href = data.getElementsByTagName('article_url')[i].firstChild.nodeValue;
        const linkText = document.createTextNode(data.getElementsByTagName('article_title')[i].firstChild.nodeValue);
        results.appendChild(newLink).appendChild(linkText);
        const lineBreak = document.createElement('br');
        results.appendChild(lineBreak);
        
      }
    
    }

    // if results flag is never flipped from false to true,
    // then no results were found
    // display the 'No results were found' message
    if (!ls.resultsFound) {
      results.innerHTML = 'No results were found.';
    }
  
  },

  sendRequest : function(url, func, postData) {

    const xhr = new XMLHttpRequest();
    if (!xhr) { return; }
    const method = (postData) ? "POST" : "GET";
    xhr.open(method, url, true);
    if (postData) {
      xhr.setRequestHeader('Content-Type','application/x-www-form-urlencoded');
    }
    xhr.onreadystatechange = function() {
      if (xhr.readyState !== 4) { return; }
      if (xhr.status !== 200 && xhr.status !== 304) {
        alert('HTTP error ' + xhr.status); return;
      }
      func(xhr);
    }
    if (xhr.readyState === 4) { return; }
    xhr.send(postData);

  }

}

ls.init();

See how this live search example renders

Considerations with the Live Search Example

• getElementsByTagName() works just fine in XML documents.
• getAttribute() and setAttribute() will also work fine; dot notation and subscript notation will not work with XML documents (they only work with the HTML DOM).
• getElementById() will not work unless we have defined a Document Type Definition (DTD) in which an attribute has been given a type of id. We are not using a DTD in our XML data file so the method will always return null
• If we wanted to get at the root element in the XML data we can use document.documentElement
• To prevent browsers from caching the data file (in which case updated versions do not get downloaded) we append a random number as a parameter. Not the cleanest of approaches, but necessary. Servers discard / ignore that extraneous data in the requested URL.
• Alternatives to the random number would be setting the cache-control and pragma headers to no-cache or setting the expires header to a past date or an invalid date, such as -1.

The JSON Object

• This object has two static methods.
• JSON.parse() is passed a text string (e.g., '{"success":false, "total":5}') and transforms it into an Object object, array object, or something else (string, number, boolean, null). If you have an Object object, you can access its data properties using dot notation.
• JSON.stringify() goes the other direction, converting an Object object to a text string.
• When sending off data to an online service's API, you would stringify your data first because whatever you send around the web is usually a string. And when you receive data from some other website / web application API, you would parse it so that your code can work with it.

Updating News Headings (JSON Data)

Structure (json_news.html)

<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
  <title>XHR News Heading Example</title>
  <meta charset="utf-8" />
  <link rel="stylesheet" href="news.css" />
</head>
<body>
<div id="newsSection" aria-live="polite"></div>

<script src="json_news.js"></script>
</body>
</html>

Presentation (json_news.css)

body, h1 {font: 90%/1.3 verdana, sans-serif;}
h1 {font-size: 115%; font-style: italic; margin: 0 0 5px;}
#newsSection {padding: 5px; width: 300px; border: 3px double #ccc;}

Data File (news_data.json)

{ 
  "newsItems" : [

  { "heading" : "News Item 1", 
    "url" : "http://www.news.com/1/", 
    "summary" : "Sample text sample text sample text sample text" },
  
  { "heading" : "News Item 2", 
    "url" : "http://www.news.com/2/", 
    "summary" : "Sample text sample text sample text sample text" },
  
  { "heading" : "News Item 3", 
    "url" : "http://www.news.com/3/", 
    "summary" : "Sample text sample text sample text sample text" },
  
  { "heading" : "News Item 4", 
    "url" : "http://www.news.com/4/", 
    "summary" : "Sample text sample text sample text sample text" },
  
  { "heading" : "News Item 5", 
    "url" : "http://www.news.com/5/", 
    "summary" : "Sample text sample text sample text sample text" },
  
  { "heading" : "News Item 6", 
    "url" : "http://www.news.com/6/", 
    "summary" : "Sample text sample text sample text sample text" },
  
  { "heading" : "News Item 7", 
    "url" : "http://www.news.com/7/", 
    "summary" : "Sample text sample text sample text sample text" },
  
  { "heading" : "News Item 8", 
    "url" : "http://www.news.com/8/", 
    "summary" : "Sample text sample text sample text sample text" }

  ]
}

Behavior (json_news.js)

"use strict";

const news = {

  // reference to body tag
  theBody : document.querySelector('body'),

  // reference to container for news
  newsHolder : document.querySelector('#newsSection'),

  init : function() {
    
    // send the request for the JSON data
    this.sendRequest('news_data.json?' + Math.random(), this.populateNews);

    // request the data every 30 seconds; 
    // in reality the data updates would
    // be more spread out but that is a pain to test
    const newsRefresh = setInterval(this.pullNewData, 30000);

  },

  // function called periodically to refresh the data
  pullNewData : function() {
  
    news.sendRequest('news_data.json?' + Math.random(), news.populateNews);
  
  },

  populateNews : function(xhr) {

    // convert JSON data into a JavaScript object
    const data = JSON.parse(xhr.responseText);
    const totalNewsItems = data.newsItems.length;

    let str = '<h1>Latest News</h1>';

    for (let i=0; i<totalNewsItems; i++) {
      
      str += '<p><a href="' + data.newsItems[i].url + '">';
      str += data.newsItems[i].heading + '</a><br />';
      str += data.newsItems[i].summary + '</p>';

    }
    
    // append the news container into the body
    news.newsHolder.innerHTML = str;

  },

  sendRequest : function(url, func, postData) {

    const xhr = new XMLHttpRequest();
    if (!xhr) { return; }
    const method = (postData) ? "POST" : "GET";
    xhr.open(method, url, true);
    if (postData) {
      xhr.setRequestHeader('Content-Type','application/x-www-form-urlencoded');
    }
    xhr.onreadystatechange = function() {
      if (xhr.readyState !== 4) { return; }
      if (xhr.status !== 200 && xhr.status !== 304) {
        alert('HTTP error ' + xhr.status); return;
      }
      func(xhr);
    }
    if (xhr.readyState === 4) { return; }
    xhr.send(postData);

  }

}

news.init();

See how this news example renders

Considerations with the Updating News Headlines Example

• The Network tab in the Developer Tools will show the activity every 30 seconds as a call is made and that part of the page is updated.
• If this was hooked up to a dynamic data source (likely a server-side script querying a database) then we would consider additional enhancements, such as a status message with role="status" during the updates (assuming that the data was changing and the message was warranted).