Instrument and measure XMLHttpRequest
(AJAX) requests.
With this plugin, sites can measure the performance of XMLHttpRequests
(XHRs) and other in-page interactions after the page has been loaded.
This plugin also monitors DOM manipulations following a XHR to filter out "background" XHRs.
This plugin provides the backbone for the BOOMR.plugins.SPA
plugin. Single Page App navigations
use XHR and DOM monitoring to determine when the SPA navigations are complete.
This plugin has a corresponding Header Snippets that helps monitor XHRs prior to Boomerang loading.
For information on how to include this plugin, see the Building tutorial.
What is Measured
When AutoXHR
is enabled, this plugin will monitor several events:
XMLHttpRequest
requestsFetch
API requests- Clicks
window.History
changes indirectly through SPA plugins (History, Angular, etc.)
When any of these events occur, AutoXHR
will start monitoring the page for
other events, DOM manipulations and other networking activity.
As long as the event isn't determined to be background activity (i.e an XHR that didn't change the DOM at all), the event will be measured until all networking activity has completed.
This means if your click generated an XHR that triggered an updated view to fetch more HTML that added images to the page, the entire event will be measured from the click to the last image completing.
Usage
To enable AutoXHR, you should set instrument_xhr
to true
:
BOOMR.init({
instrument_xhr: true
});
Once enabled and initialized, the window.XMLHttpRequest
object will be
replaced with a "proxy" object that instruments all XHRs.
Monitoring XHRs
After AutoXHR
is enabled, any XMLHttpRequest.send
will be monitored:
xhr = new XMLHttpRequest();
xhr.open("GET", "/api/foo");
xhr.send(null);
If this XHR triggers DOM changes, a beacon will eventually be sent.
This beacon will have http.initiator=xhr
and the beacon parameters will differ
from a Page Load beacon. See BOOMR.plugins.RT
and
BOOMR.plugins.NavigationTiming
for details.
Combining XHR Beacons
By default AutoXHR
groups all XHR activity that happens in the same event together.
If you have one XHR that immediately triggers a second XHR, you will get a single
XHR beacon. The u
(URL) will be of the first XHR.
If you don't want this behavior, and want to measure every XHR on the page, you
can enable alwaysSendXhr=true
. When set, every
distinct XHR will get its own XHR beacon.
BOOMR.init({
AutoXHR: {
alwaysSendXhr: true
}
});
alwaysSendXhr
can also be a list of strings
(matching URLs), regular expressions (matching URLs), or a function which returns
true for URLs to always send XHRs for.
BOOMR.init({
AutoXHR: {
alwaysSendXhr: [
"domain.com",
/regexmatch/,
]
}
});
// or
BOOMR.init({
AutoXHR: {
alwaysSendXhr: function(url) {
return url.indexOf("domain.com") !== -1;
}
}
});
Compatibility and Browser Support
Currently supported Browsers and platforms that AutoXHR will work on:
- IE 9+ (not in quirks mode)
- Chrome 38+
- Firefox 25+
In general we support all browsers that support MutationObserver and XMLHttpRequest.
We will not use MutationObserver in IE 11 due to several browser bugs.
See BOOMR.utils.isMutationObserverSupported
for details.
Excluding Certain Requests or DOM Elements From Instrumentation
Whenever Boomerang intercepts an XMLHttpRequest
(or Fetch), it will check if that request
matches anything in the XHR exclude list. If it does, Boomerang will not
instrument, time, send a beacon for that request, or include it in the
BOOMR.plugins.SPA
calculations.
There are two methods of excluding XHRs: Defining the BOOMR.xhr_excludes
array,
or using the excludeFilters
option.
The BOOMR.xhr_excludes
XHR excludes list is defined by creating a map, and
adding URL parts that you would like to exclude from instrumentation. You
can put any of the following in BOOMR.xhr_excludes
:
- A full HREF
- A hostname
- A path
Example:
BOOMR = window.BOOMR || {};
BOOMR.xhr_excludes = {
"www.mydomain.com": true,
"a.anotherdomain.net": true,
"/api/v1/foobar": true,
"https://mydomain.com/dashboard/": true
};
The excludeFilters
gives you more control by allowing you
to specify one or more callbacks that will be run for each XHR/Fetch. If any callback
returns true
, the XHR/Fetch will not be instrumented.
BOOMR.init({
AutoXHR: {
excludeFilters: [
function(anchor) {
return anchor.href.match(/non-trackable/);
}
]
}
});
Finally, the domExcludeFilters
DOM filters can be used to filter out
specific DOM elements from being tracked (marked as "uninteresting").
BOOMR.init({
AutoXHR: {
domExcludeFilters: [
function(elem) {
return elem.id === "ignore";
}
]
}
});
Beacon Parameters
XHR beacons have different parameters in general than Page Load beacons.
- Many of the timestamps will differ, see
BOOMR.plugins.RT
- All of the
nt_*
parameters are ResourceTiming, seeBOOMR.plugins.NavigationTiming
u
: the URL of the resource that was fetchedpgu
: The URL of the page the resource was fetched onhttp.initiator
:xhr
for both XHR and Fetch requests
Interesting Nodes
A MutationObserver
is used to detect "interesting" nodes. Interesting nodes are
new IMG/IMAGE/IFRAME/LINK (rel=stylesheet) nodes or existing nodes that are
changing their source URL.
We consider the following "uninteresting" nodes:
- Nodes that have either a width or height <= 1px.
- Nodes with either a width or height of 0px.
- Nodes that have display:none.
- Nodes that have visibility:hidden.
- Nodes with an opacity:0.
- Nodes that update their source URL to the same value.
- Nodes that have a blank source URL.
- Images with a loading="lazy" attribute
- Nodes that have a source URL starting with
about:
,javascript:
ordata:
. - SCRIPT nodes because there is no consistent way to detect when they have loaded.
- Existing IFRAME nodes that are changing their source URL because is no consistent way to detect when they have loaded.
- Nodes that have a source URL that matches a AutoXHR exclude filter rule.
- Nodes that have been manually excluded
Algorithm
Here's how the general AutoXHR algorithm works:
-
0.0
SPA hard route change (page navigation)- Monitor for XHR resource requests and interesting Mutation resource requests for 1s or at least until page onload. We extend our 1s timeout after all interesting resource requests have completed.
-
0.1
SPA soft route change from a synchronous call (eg. History changes as a result of a pushState or replaceState call)- In this case we get the new URL when the developer calls pushState or replaceState.
- We create a pending event with the start time and the new URL.
- We do not know if they plan to make an XHR call or use a dynamic script node, or do nothing interesting (eg: just make a div visible/invisible).
- We also do not know if they will do this before or after they've called pushState/replaceState.
- Our best bet is to monitor if either a XHR resource requests or interesting Mutation resource requests will happen in the next 1s.
- When interesting resources are detected, we wait until they complete.
- We restart our 1s timeout after all interesting resources have completed.
-
If something uninteresting happens, we set the timeout for 1 second if it wasn't already started.
- We'll only do this once per event since we don't want to continuously extend the timeout with each uninteresting event.
-
If nothing happens during the additional timeout, we stop watching and fire the event. The event end time will be end time of the last tracked resource.
-
If nothing interesting was detected during the first timeout and the URL has not changed then we drop the event.
-
0.2
SPA soft route change from an asynchronous call (eg. History changes as a result of the user hitting Back/Forward and we get a window.popstate event)- In this case we get the new URL from location.href when our event listener runs.
- We do not know if this event change will result in some interesting network activity or not.
- We do not know if the developer's event listener has already run before ours or if it will run in the future or even if they do have an event listener.
- Our best bet is the same as 0.1 above.
-
0.3
SPA soft route change from a click that triggers a XHR before the state is changed (whenspaStartFromClick
is enabled).- We store the time of the click
- If any additional XHRs come next, we track those
- When a pushState comes after the XHRs (before the timeout), we will "migrate" the click to a SPA event
-
1
Click initiated (Only available when no SPA plugins are enabled)- User clicks on something.
- We create a pending event with the start time and no URL.
- We turn on DOM observer, and wait up to 50 milliseconds for activity.
- If nothing happens during the first timeout, we stop watching and clear the event without firing it.
- Else if something uninteresting happens, we set the timeout for 1s
if it wasn't already started.
- We'll only do this once per event since we don't want to continuously extend the timeout with each uninteresting event.
- Else if an interesting node is added, we add load and error listeners
and turn off the timeout but keep watching.
- Once all listeners have fired, we start waiting again up to 50ms for activity.
- If nothing happens during the additional timeout, we stop watching and fire the event.
-
2
XHR/Fetch initiated- XHR or Fetch request is sent.
- We create a pending event with the start time and the request URL.
- We watch for all changes in XHR state (for async requests) and for load (for all requests).
- We turn on DOM observer at XHR Onload or when the Fetch Promise resolves.
We then wait up to 50 milliseconds for activity.
- If nothing happens during the first timeout, we stop watching and clear the event without firing it.
- If something uninteresting happens, we set the timeout for 1 second if
it wasn't already started.
- We'll only do this once per event since we don't want to continuously extend the timeout with each uninteresting event.
- Else if an interesting node is added, we add load and error listeners
and turn off the timeout.
- Once all listeners have fired, we start waiting again up to 50ms for activity.
- If nothing happens during the additional timeout, we stop watching and fire the event.
What about overlap?
-
3.1
XHR/Fetch initiated while a click event is pending- If first click watcher has not detected anything interesting or does not have a URL, abort it
- If the click watcher has detected something interesting and has a URL, then
- Proceed with 2 above.
- Concurrently, click stops watching for new resources
- Once all resources click is waiting for have completed then fire the event.
-
3.2
Click initiated while XHR/Fetch event is pending- Ignore click
-
3.3
Click initiated while a click event is pending- If first click watcher has not detected anything interesting or does not have a URL, abort it.
- Else proceed with parallel event steps from 3.1 above.
-
3.4
XHR/Fetch initiated while an XHR/Fetch event is pending- Add the second XHR/Fetch as an interesting resource to be tracked by the XHR pending event in progress.
-
3.5
XHR/Fetch initiated while SPA event is pending- Add the second XHR/Fetch as an interesting resource to be tracked by the XHR pending event in progress.
-
3.6
SPA event initiated while an XHR event is pending- Proceed with 0 above.
- Concurrently, XHR event stops watching for new resources. Once all resources the XHR event is waiting for have completed, fire the event.
-
3.7
SPA event initiated while a SPA event is pending- If the pending SPA event had detected something interesting then send an aborted SPA beacon. If not, drop the pending event.
- Proceed with 0 above.
Methods
-
addDomExcludeFilter(cb, ctx [, name])
-
Add a DOM filter function to the list of functions to run to validate if an DOM element should be instrumented.
Parameters:
Name Type Argument Description cb
BOOMR.plugins.AutoXHR.htmlElementCallback Callback to run to validate filtering of an XHR Request
ctx
Object Context to run {@param cb} in
name
string <optional>
Optional name for the filter, called out when running exclude filters for debugging purposes
Example
BOOMR.plugins.AutoXHR.addDomExcludeFilter(function(elem) { return elem.id === "ignore"; }, null, "exampleFilter");
-
addExcludeFilter(cb, ctx [, name])
-
Add a XHR filter function to the list of functions to run to validate if an XHR should be instrumented.
Parameters:
Name Type Argument Description cb
BOOMR.plugins.AutoXHR.htmlElementCallback Callback to run to validate filtering of an XHR Request
ctx
Object Context to run {@param cb} in
name
string <optional>
Optional name for the filter, called out when running exclude filters for debugging purposes
Example
BOOMR.plugins.AutoXHR.addExcludeFilter(function(anchor) { var m = anchor.href.match(/some-page\.html/g); // If matching flag to not instrument if (m && m.length > 0) { return true; } return false; }, null, "exampleFilter");
-
enableAutoXhr()
-
Enables AutoXHR if not already enabled.
-
getMutationHandler()
-
Gets the
MutationHandler
Returns:
Type: MutationHandler
Handler
-
getPathName(anchor)
-
Tries to resolve
href
links from relative URLs.This implementation takes into account a bug in the way IE handles relative paths on anchors and resolves this by assigning
a.href
to itself which triggers the URL resolution in IE and will fix missing leading slashes if necessary.Parameters:
Name Type Description anchor
string The anchor object to resolve
Returns:
Type: string
The unrelativized URL href
-
init(config)
-
Initializes the plugin.
Parameters:
Name Type Description config
object Configuration
Properties
Name Type Argument Default Description instrument_xhr
boolean <optional>
Whether or not to instrument XHR
AutoXHR.spaBackEndResources
Array.<string> <optional>
Default resources to count as Back-End during a SPA nav
AutoXHR.monitorFetch
boolean <optional>
Whether or not to instrument fetch()
AutoXHR.fetchBodyUsedWait
number <optional>
If the fetch response's bodyUsed flag is false, we'll wait this amount of ms before checking RT for an entry. Setting to 0 will disable this feature
AutoXHR.alwaysSendXhr
boolean | Array.<string> | Array.<RegExp> | Array.<function()> <optional>
Whether or not to send XHR beacons for every XHR.
captureXhrRequestResponse
boolean <optional>
Whether or not to capture an XHR's request and response bodies on for the
xhr_load
event.AutoXHR.spaIdleTimeout
number <optional>
1000 Timeout for Single Page Applications after the final resource fetch has completed before calling the SPA navigation complete. Default is 1000ms.
AutoXHR.xhrIdleTimeout
number <optional>
50 Timeout for XHRs after final resource fetch has completed before calling the XHR complete. Default is 50ms.
AutoXHR.xhrRequireChanges
boolean <optional>
true Whether or not a XHR beacon will only be triggered if there were DOM changes.
AutoXHR.spaStartFromClick
boolean <optional>
false In Single Page Apps, start tracking the SPA Soft Navigation from any preceeding clicks. If false, will start from the most recent pushState.
Returns:
BOOMR.plugins.AutoXHR
The AutoXHR plugin for chaining -
is_complete()
-
This plugin is always complete (ready to send a beacon)
Returns:
Type: boolean
true
-
loadFinished(resource)
-
Mark this as the time load ended via resources loadEventEnd property, if this resource has been added to the
MutationHandler
already notify that the resource has finished. Otherwise add this call to the lise of Events that occured.Parameters:
Name Type Description resource
object Resource
-
shouldExcludeXhr(anchor)
-
Based on the contents of BOOMR.xhr_excludes check if the URL that we instrumented as XHR request matches any of the URLs we are supposed to not send a beacon about.
Parameters:
Name Type Description anchor
HTMLAnchorElement HTML anchor element with URL of the element checked against
BOOMR.xhr_excludes
Returns:
Type: boolean
true
if intended to be excluded,false
if it is not in the list of excludables
Type Definitions
-
htmlElementCallback(elem)
-
A callback with a HTML element.
Parameters:
Name Type Description elem
HTMLAnchorElement HTML element