Scrape Table Data from the Web

There is a wealth of data available on the web that dynamically updates, but is not published as a neat .csv file that works with eagle.io out of the box.

An example of this type of data is river heights published by BOM in near real-time in a HTLM table http://www.bom.gov.au/fwo/IDQ65388/IDQ65388.540319.tbl.shtml

This article shows you how to import this data to eagle.io using Google Sheets as middle ware.

 

River height monitoring - HTML Table that updates up to every 2 minutes

Step 1 — Import data to google sheets using IMPORTHTML function.

IMPORTHTML(url, query, index)

  • url - The URL of the page to examine, including protocol (e.g. http://).
  • The value for url must either be enclosed in quotation marks or be a reference to a cell containing the appropriate text.
  • query - Either "list" or "table" depending on what type of structure contains the desired data. Also enclosed in quotation marks.
  • index - The index, starting at 1, which identifies which table or list or table to be returned.

 

Step 2 — We’ll configure our Spreadsheet to publish the data; this allowseagle.ioto access the information collected by our Form.

 

Step 3- Choose the specific sheet with data (default is “Entire Document”) and CSV as the format and copy the URL to the published data.

 

Back to Blog