A web application that parses statistics data from Heritrix crawl reports and displays them in charts. Built using a GWT-based paradigm as a proof of concept.
Intended for use by the NLA web archiving team only.
To add the statistics data for a new WDH harvest:
- Copy the hosts-report.txt file into <odin_data_directory>/source_data/harvest_host_reports/ and rename it yyyy_MM.txt based on the year and month of the harvest (for example 2024_07.text).
- Copy the mimetype-report.txt file into <odin_data_directory>/source_data/harvest_mime_reports/ and rename it yyyy_MM.txt based on the year and month of the harvest (for example 2024_07.text).
- Refresh the web page to view the new data. The first time the new data is loaded, it will be parsed from the report files and cached as statistics data, so it may take a minute to load. After successfully loading the first time, loading will be almost instantaneous. Note: geolocation parsing may take several minutes.
To develop ODIN, install the latest version of the GWT plugin for your IDE.
- Run the odin-server project (OdinServer.java) as a Java Application. Changes to server-side code will require this process to be restarted.
- Run the odin-ui project as a GWT web application, with Super DEV Mode enabled. Example eclipse run configuration below:
- In UIClientContext.java, change the variable DEV to true.
- Navigate to http://127.0.0.1:8888/ in your browser.
- Any changes made to UI code only requires the browser to be refreshed.