Visualize HTML with Infinity
📊 Overview
Caution
HTML query type should be used only for retrieving data from legacy systems where there are no alternative APIs exist. Instead of HTML query type, we strongly recommend to use other query types such as JSON, CSV, XML.
In the following example, you are going to retrieve data from a sample HTML page.
In the Query editor, fill the following query details:
- Select HTML as query type.
- Select Default (frontend) as the parser.
- Select URL as the source.
- Select GET as the HTTP method.
- Enter
https://github.com/grafana/grafana-infinity-datasource/blob/main/testdata/users.html
in the URL field of the query.
After the initial setup is done, you need to configure the selectors:
- In the root selector, you need to provide the selector which will give you array of symmetrical elements. (This can potentially be rows in a table or repeating div elements with symmetrical structure). In our case, we are entering
table:nth-child(1) tbody tr
(CSS selector) as our root selector. Alternatively, you can givetr
as selector if your HTML content has only one table. Also, if the table has any unique selectors such as ID, use that as the selector instead. - From our HTML structure, we know that each row contains several div elements with each div representing a property of the user, so we need to uniquely identify the div elements corresponding to the user property.
- Add a column and enter
td:nth-child(1)
as selector field. Also mark this asName
in the as/alias field. We can leave this as a string - Add another column and enter
td:nth-child(2)
as selector field. Also mark this asAge
in the as/alias field. We know that this is a number, so we can change the field type to number. - Add another column and enter
td:nth-child(3)
as selector field. Also mark this asCountry
in the as/alias field - Add any other columns you may need.
An example of the above query can be found in play.grafana site for reference.
Limitations
- Only symmetrical data can be queries. (Example:
table
withcolspan
orrowspan
will break the scrapping) - We only support text element querying. Retrieving other HTML attributes is not supported.
- If you prefer to use backend parser for HTML query type, be aware that the backend HTML query parser is experimental and subject to breaking changes. Also, only the HTML pages compatible to XML syntax can be used with HTML backend query type.
- If the scrapping is at high frequency/refresh rate, websites may block you or your IP address. Be sensible and responsible about setting your refresh limits.
- Caching is not implemented, so be aware of the rate limits.