Home | Javadocs | Web-Scraping Tutorial | JSON Querying Tutorial | FAQ | Download

Jaunt Java Web Scraping & JSON Querying

Introduction
Oct 31, 2024
Jaunt 1.6.1 release
Test drive Jaunt today and leave feedback in the forum to help shape the next release!
Jaunt is a Java library for web-scraping, web-automation and JSON querying. The library provides a fast, ultra-light browser that is "headless" (ie has no GUI). The browser provides web-scraping functionality, access to the DOM, and control over each HTTP Request/Response, but does not support Javascript*. Jaunt enables your Java programs to:
  • peform web-scraping and JSON data extraction
  • work with forms and tables
  • control/process individual HTTP Requests/Responses
  • interface with REST APIs or web-apps (JSON, HTML, XHTML, or XML).
*Now you can automate Chrome, Firefox, Safari, Internet Explorer, etc,
with full Javacript support, using Jauntium.
Code example: Google scraper - search for 'butterflies'
GoogleScraperDemo.java: 
UserAgent userAgent = new UserAgent();         //create new userAgent (headless browser)
userAgent.visit("http://google.com");          //visit google
userAgent.doc.apply("butterflies").submit();   //apply form input and submit
    
Elements links = userAgent.doc.findEvery("<h3>").findEvery("<a>");  //find search result links
for(Element link : links) System.out.println(link.getAt("href"));   //print results
Features:
Jaunt is free [see product comparison]. Features include:
  • Parsing dirty HTML, XHTML, XML, JSON
  • Support for HTTP, HTTPS & basic auth.
  • Form fill-out via field labels/names/sequence.
  • Generating form data permutations.
  • File downloading/uploading.
  • Saving complete web page (images, js, css, etc).
  • Table data extraction.
  • Fluent DOM navigation & search (search chaining).
  • Regex-enabled querying in DOM & JSON.
  • HTTP header/cookie manipulation.
  • HTTP/HTTPS proxy support.
  • Customizable caching & content handlers.
  • Web pagination discovery.
  • 100% Java (no dependencies)
Webscraping Tutorial (Quickstart)
  1. Create a UserAgent, visit a url, print the HTML.
  2. UserAgent settings, searching using findFirst.
  3. Opening HTML from a String, retrieving an Element's text.
  4. Accessing an Element's attributes/properties.
  5. Opening HTML from a file, accessing inner/outerHTML and inner/outerXML.
  6. Detecting HTTP errors and connection errors with the Response object.
  7. Handling HTTP errors, connection errors, and search Exceptions.
  8. Searching using findEach, iterating through search results.
  9. Searching using findEvery vs. findEach
  10. Searching using getElement and getEach and
    Search method summary.
  11. Searching with regular expressions and
    Tag query syntax
  12. Filling-out form fields in sequence using Document.apply().
  13. Filling-out form fields by label with the Document object (textfields, password fields, checkboxes).
  14. Filling-out form fields by label with the Document object (select fields, textarea fields, radiobuttons).
  15. Filling-out form fields by name with the Form object (textfields, password fields, checkboxes, menus, textareas, radiobuttons).
  16. Generating a form's request permutations.
  17. Table traversal.
  18. Table text extraction using the Table component.
  19. Table cell extraction using the Table component.
  20. Pagination Discovery.
  21. Using the HTML cache.
Web Automation Tutorial - Extra Topics
  1. Iterating through an Element's child Nodes.
  2. Accessing Comments, DOCTYPE definitions, processing instructions, and CDATA.
  3. Using Filters to block/allow content when parsing.
  4. Injecting/Overriding HTTP Request Headers.
  5. Sending HEAD, POST, PUT, & DELETE Requests.
  6. Using content handlers to retrieve JS/CSS/GIF/etc files.
  7. Using an HTTP proxy (UserAgent-level config).
  8. Using an HTTP proxy with username/password (System-level config).
  9. Using an HTTPS proxy (System-level config).
JSON Querying Tutorial (Quickstart)
  1. Create a UserAgent, send a GET request, print JSON response.
  2. UserAgent settings, searching using findFirst.
  3. Detecting HTTP errors and connection errors with the Response object.
  4. Opening JSON from a file, accessing a JNode's type and parent.
  5. Opening JSON from a String, working with JSON arrays.
  6. Searching using findEvery, search chaining.
  7. Searching using findEach vs findEvery.
  8. Searching using getEach, query syntax, search method summary.
  9. Searching with regular expressions.
  10. Detecting HTTP errors and connection errors with the Response object.
Home | Javadocs | Web-Scraping Tutorial | JSON Querying Tutorial | FAQ | Download