Merge pull request #18 from serpapi/bump-up-to-0.2.5

Bump up the version to 0.2.5

Gemfile

@@ -2,6 +2,9 @@
 
 source "https://rubygems.org"
 
+gemspec
+
 gem 'nokolexbor'
 gem 'http'
-gem 'parallel', '~> 1.20', '>= 1.20.1'
+gem 'parallel', '~> 1.20', '>= 1.20.1'
+gem 'json'

Gemfile.lock

@@ -1,3 +1,12 @@
+PATH
+  remote: .
+  specs:
+    google-local-results-ai-parser (0.2.4)
+      http
+      json
+      nokolexbor
+      parallel (~> 1.20, >= 1.20.1)
+
 GEM
   remote: https://rubygems.org/
   specs:
@@ -17,6 +26,7 @@ GEM
     http-cookie (1.0.5)
       domain_name (~> 0.5)
     http-form_data (2.3.0)
+    json (2.6.3)
     llhttp-ffi (0.4.0)
       ffi-compiler (~> 1.0)
       rake (~> 13.0)
@@ -32,9 +42,11 @@ PLATFORMS
   ruby
 
 DEPENDENCIES
+  google-local-results-ai-parser!
   http
+  json
   nokolexbor
-  parallel
+  parallel (~> 1.20, >= 1.20.1)
 
 BUNDLED WITH
    2.4.7

README.md

@@ -49,10 +49,6 @@ $ gem install google-local-results-ai-parser
 To use the `google-local-results-ai-parser` gem, you need to include the necessary dependencies:
 ```rb
 require 'google-local-results-ai-parser'
-require 'nokolexbor'
-require 'http'
-require 'parallel'
-require 'json'
 ```
 
 <h3 align="center">Parsing HTML</h3>
@@ -133,6 +129,9 @@ The gem provides some configuration options that you can customize according to
     - **Known clashes**: Sometimes the model serves clashing results for one label. This may happen due to limitations of the model, the generality of meaning of the classified text, or the limitations of the dataset model was trained on. The gem can clear out the majority of the known clashes, and correct them using traditional logical algorithms.
     - **General clashes**: The model can compare the assurance score of two texts with same label, and pick the one with a higher score to automatically correct the results after predictions. From raw observations, doing after-corrections only once is observed to be enough. You may increase `iteration` parameter to force the after-corrections more in case of any further clashes.
 
+- `debug`: The parameter allows the returning of debugging time information needed to calculate maximum time it takes to connect to the server. You may take a look at [example_debug.rb](https://github.com/serpapi/google-local-results-ai-parser/blob/master/example_debug.rb) and [example_multiple_debug.rb](https://github.com/serpapi/google-local-results-ai-parser/blob/master/example_debug.rb) for reference.
+
+- `no_cache`: The parameter forces the model endpoint to make a prediciton instead of serving cached results. This is also used for debugging purposes to see the initial load of the model. You may take a look at [example_debug.rb](https://github.com/serpapi/google-local-results-ai-parser/blob/master/example_debug.rb) and [example_multiple_debug.rb](https://github.com/serpapi/google-local-results-ai-parser/blob/master/example_multiple_debug.rb) for reference.
 
 [gem-shield]: https://img.shields.io/gem/v/google-local-results-ai-parser.svg
 [gem-url]: https://rubygems.org/gems/google-local-results-ai-parser

example.rb

@@ -1,8 +1,12 @@
 require './lib/google-local-results-ai-parser'
 
 html = '<div class="VkpGBb"><div class="cXedhc"><a class="vwVdIc wzN8Ac rllt__link a-no-hover-decoration" jsname="kj0dLd" data-cid="3982431987169598286" jsaction="click:h5M12e;" role="link" tabindex="0" data-ved="2ahUKEwiyjZWz2L3_AhVyRfEDHX0cB7AQlrABegQIBBAE"><div><div class="rllt__details"><div class="dbg0pd" aria-level="3" role="heading"><span class="OSrXXb">A.D.A. Auto Repair Center</span></div><div><span><span class="Y0A0hc"><span class="yi40Hd YrbPuc" aria-hidden="true">4.9</span><span class="z3HNkc" aria-label="Rated 4.9 out of 5," role="img"><span style="width:70px"></span></span><span class="RDApEe YrbPuc">(29)</span></span></span> · Vehicle repair shop</div><div>30+ years in business · Chilis 18, Nicosia, Cyprus</div><div>Open ⋅ Closes 6:30 pm · 99 857782</div></div></div></a></div><a class="yYlJEf Q7PwXb L48Cpd brKmxb" aria-describedby="tsuid_7" href="https://www.googleadservices.com/pagead/aclk?sa=L&amp;ai=DChcSEwiKrqCz2L3_AhUSydUKHcd_CsUYABAAGgJ3cw&amp;ohost=www.google.com&amp;cid=CAESauD2xDTWkwyZYLj4k4wJQMqIa8OsgCSH_ZtFtUchveo_Se0DYkOBYrvz6g_0igL0zZIhTSFBXYV76Y5WgwxcjvlgFKTql7_YjvY4jVkOgn3AUIGwBdEZ3oO9cT-O9gU4B8fLVw8cMFrimpM&amp;sig=AOD64_3UrhBGCtSgNdtf4HAVNamKS0rgvg&amp;q=&amp;ctype=99&amp;ved=2ahUKEwiyjZWz2L3_AhVyRfEDHX0cB7AQhKwBegQIBBAO&amp;adurl=" data-ved="2ahUKEwiyjZWz2L3_AhVyRfEDHX0cB7AQhKwBegQIBBAO"><div class="wLAgVc"><span class="XBBs5 z1asCe GYDk8c"><svg focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-1 17.93c-3.95-.49-7-3.85-7-7.93 0-.62.08-1.21.21-1.79L9 15v1c0 1.1.9 2 2 2v1.93zm6.9-2.54c-.26-.81-1-1.39-1.9-1.39h-1v-3c0-.55-.45-1-1-1H8v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41c2.93 1.19 5 4.06 5 7.41 0 2.08-.8 3.97-2.1 5.39z"></path></svg></span><div class="BSaJxc">Website</div></div></a><a style="cursor:pointer" data-url="//www.googleadservices.com/pagead/aclk?sa=L&amp;ai=DChcSEwiKrqCz2L3_AhUSydUKHcd_CsUYABABGgJ3cw&amp;ohost=www.google.com&amp;cid=CAESauD2xDTWkwyZYLj4k4wJQMqIa8OsgCSH_ZtFtUchveo_Se0DYkOBYrvz6g_0igL0zZIhTSFBXYV76Y5WgwxcjvlgFKTql7_YjvY4jVkOgn3AUIGwBdEZ3oO9cT-O9gU4B8fLVw8cMFrimpM&amp;sig=AOD64_2KqinRQ9g6leNprF0lTF4Fd9V0Vg&amp;adurl=&amp;ctype=50&amp;q=" href="#" jsaction="trigger.Ez7VMc" tabindex="0" class="yYlJEf VByer Q7PwXb VDgVie brKmxb" aria-describedby="tsuid_7" data-ved="2ahUKEwiyjZWz2L3_AhVyRfEDHX0cB7AQhawBegQIBBAP"><div><span class="TU05kd"></span><div class="UbRuwe">Directions</div></div></a></div>'
-bearer_token = 'Huggingface Token or Private Server Key'
+bearer_token = 'Huggingface Token' # Without the word Bearer
 
 results = GoogleLocalResultsAiParser.parse(html: html, bearer_token: bearer_token)
 
 puts results
+
+# This is an example code to show you how you can make single parsing
+# using Free Inference API of Huggingface. Beware that the endpoint
+# is ratelimited.

example_debug.rb

@@ -0,0 +1,19 @@
+require './lib/google-local-results-ai-parser'
+
+html = '<div class="VkpGBb"><div class="cXedhc"><a class="vwVdIc wzN8Ac rllt__link a-no-hover-decoration" jsname="kj0dLd" data-cid="3982431987169598286" jsaction="click:h5M12e;" role="link" tabindex="0" data-ved="2ahUKEwiyjZWz2L3_AhVyRfEDHX0cB7AQlrABegQIBBAE"><div><div class="rllt__details"><div class="dbg0pd" aria-level="3" role="heading"><span class="OSrXXb">A.D.A. Auto Repair Center</span></div><div><span><span class="Y0A0hc"><span class="yi40Hd YrbPuc" aria-hidden="true">4.9</span><span class="z3HNkc" aria-label="Rated 4.9 out of 5," role="img"><span style="width:70px"></span></span><span class="RDApEe YrbPuc">(29)</span></span></span> · Vehicle repair shop</div><div>30+ years in business · Chilis 18, Nicosia, Cyprus</div><div>Open ⋅ Closes 6:30 pm · 99 857782</div></div></div></a></div><a class="yYlJEf Q7PwXb L48Cpd brKmxb" aria-describedby="tsuid_7" href="https://www.googleadservices.com/pagead/aclk?sa=L&amp;ai=DChcSEwiKrqCz2L3_AhUSydUKHcd_CsUYABAAGgJ3cw&amp;ohost=www.google.com&amp;cid=CAESauD2xDTWkwyZYLj4k4wJQMqIa8OsgCSH_ZtFtUchveo_Se0DYkOBYrvz6g_0igL0zZIhTSFBXYV76Y5WgwxcjvlgFKTql7_YjvY4jVkOgn3AUIGwBdEZ3oO9cT-O9gU4B8fLVw8cMFrimpM&amp;sig=AOD64_3UrhBGCtSgNdtf4HAVNamKS0rgvg&amp;q=&amp;ctype=99&amp;ved=2ahUKEwiyjZWz2L3_AhVyRfEDHX0cB7AQhKwBegQIBBAO&amp;adurl=" data-ved="2ahUKEwiyjZWz2L3_AhVyRfEDHX0cB7AQhKwBegQIBBAO"><div class="wLAgVc"><span class="XBBs5 z1asCe GYDk8c"><svg focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-1 17.93c-3.95-.49-7-3.85-7-7.93 0-.62.08-1.21.21-1.79L9 15v1c0 1.1.9 2 2 2v1.93zm6.9-2.54c-.26-.81-1-1.39-1.9-1.39h-1v-3c0-.55-.45-1-1-1H8v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41c2.93 1.19 5 4.06 5 7.41 0 2.08-.8 3.97-2.1 5.39z"></path></svg></span><div class="BSaJxc">Website</div></div></a><a style="cursor:pointer" data-url="//www.googleadservices.com/pagead/aclk?sa=L&amp;ai=DChcSEwiKrqCz2L3_AhUSydUKHcd_CsUYABABGgJ3cw&amp;ohost=www.google.com&amp;cid=CAESauD2xDTWkwyZYLj4k4wJQMqIa8OsgCSH_ZtFtUchveo_Se0DYkOBYrvz6g_0igL0zZIhTSFBXYV76Y5WgwxcjvlgFKTql7_YjvY4jVkOgn3AUIGwBdEZ3oO9cT-O9gU4B8fLVw8cMFrimpM&amp;sig=AOD64_2KqinRQ9g6leNprF0lTF4Fd9V0Vg&amp;adurl=&amp;ctype=50&amp;q=" href="#" jsaction="trigger.Ez7VMc" tabindex="0" class="yYlJEf VByer Q7PwXb VDgVie brKmxb" aria-describedby="tsuid_7" data-ved="2ahUKEwiyjZWz2L3_AhVyRfEDHX0cB7AQhawBegQIBBAP"><div><span class="TU05kd"></span><div class="UbRuwe">Directions</div></div></a></div>'
+bearer_token = 'Huggingface Token or Private Server Key' # Without the word Bearer
+server = 'Server URL'
+
+time_start = Time.now
+results, time_taken = GoogleLocalResultsAiParser.parse(html: html, bearer_token: bearer_token, server: server, debug: true, no_cache: true)
+time_end = Time.now
+
+puts results
+puts "----"
+puts "Results"
+puts "Total Time Taken: #{time_end - time_start} seconds"
+puts "Maximum time taken in parallel requests: #{time_taken} seconds}"
+
+# This is an example code to show you how you can debug the time it takes to connect
+# to server and time it takes to parse. The model is forced to make a prediction instead
+# of serving cached results.

example_multiple.rb

@@ -4,8 +4,12 @@
   '<div jscontroller="AtSb" class="w7Dbne CR1S4b" data-record-click-time="false" id="tsuid_38" jsdata="zt2wNd;_;B40xrM WDO8Ff;_;B40xrY" jsaction="rcuQ6b:npT2md;e3EWke:kN9HDb" data-hveid="CBgQAA"><div jsname="jXK9ad" class="uMdZh tIxNaf rllt__borderless" jsaction="mouseover:UI3Kjd;mouseleave:Tx5Rb"><div class="VkpGBb"><div class="cXedhc"><a class="vwVdIc wzN8Ac rllt__link a-no-hover-decoration" jsname="kj0dLd" data-cid="6125476509096315320" jsaction="click:h5M12e;" role="link" tabindex="0" data-ved="2ahUKEwjGmqav77v_AhVPcvEDHe0ABA4Q1YkKegQIGBAB"><div><div class="rllt__details"><div class="dbg0pd" aria-level="3" role="heading"><span class="OSrXXb">Gustav Roasting Co.</span></div><div><span><span class="Y0A0hc"><span class="yi40Hd YrbPuc" aria-hidden="true">4.8</span><span class="z3HNkc" aria-label="Rated 4.8 out of 5," role="img"><span style="width:70px"></span></span><span class="RDApEe YrbPuc">(64)</span></span></span> · Coffee shop</div><div>Mehmet Akif Caddesi No:112</div><div class="dXnVAb"><span class="BI0Dve"><span><span aria-label="Takeaway">Takeaway</span></span><span class="if66xd">·</span></span><span class="BI0Dve"><span><span aria-label="No dine-in">No dine-in</span></span><span class="if66xd">·</span></span><span class="BI0Dve"><span><span aria-label="No delivery">No delivery</span></span></span></div></div></div></a><a class="uQ4NLd b9tNq wzN8Ac rllt__link a-no-hover-decoration" aria-hidden="true" tabindex="-1" jsname="kj0dLd" data-cid="6125476509096315320" jsaction="click:h5M12e;" role="link" data-ved="2ahUKEwjGmqav77v_AhVPcvEDHe0ABA4Q1YkKegQIGBAL"><g-img class="gTrj3e"><img id="pimg_1" src="https://lh5.googleusercontent.com/p/AF1QipMJBSZBgAw-jzVoRZWxuBUixiJJajyJ4ITrtslW=w167-h167-n-k-no" class="YQ4gaf zr758c wA1Bge" alt="" data-atf="4" data-frt="0" width="92" height="92"></g-img></a></div></div></div></div>',
   '<a class="vwVdIc wzN8Ac rllt__link a-no-hover-decoration" jsname="kj0dLd" data-cid="8340045413518965442" jsaction="click:h5M12e;" role="link" tabindex="0" data-ved="2ahUKEwjVnbbfqr3_AhUEZ_EDHWpjALAQyTN6BAgQEAE"><div><div class="rllt__details"><div class="dbg0pd" aria-level="3" role="heading"><span class="OSrXXb">Fusion Kitchen</span></div><div><span><span class="Y0A0hc"><span class="yi40Hd YrbPuc" aria-hidden="true">4.6</span><span class="z3HNkc" aria-label="Rated 4.6 out of 5," role="img"><span style="width:62px"></span></span><span class="RDApEe YrbPuc">(30)</span></span></span> · Fast Food</div><div>Jordan, Hong Kong</div><div><span><b class="Z5bgrc">Takeout</b>: Now ⋅ Ends 10 pm</span></div></div></div></a>',
 ]
-bearer_token = 'Huggingface Token or Private Server Key'
+bearer_token = 'Huggingface Token' # Without the word Bearer
 
 results = GoogleLocalResultsAiParser.parse_multiple(html_parts: html_parts, bearer_token: bearer_token)
 
 puts results
+
+# This is an example code to show you how you can make multiple parsings
+# using Free Inference API of Huggingface. Beware that the endpoint
+# is ratelimited.

example_multiple_debug.rb

@@ -0,0 +1,22 @@
+require './lib/google-local-results-ai-parser'
+
+html_parts = [
+  '<div jscontroller="AtSb" class="w7Dbne CR1S4b" data-record-click-time="false" id="tsuid_38" jsdata="zt2wNd;_;B40xrM WDO8Ff;_;B40xrY" jsaction="rcuQ6b:npT2md;e3EWke:kN9HDb" data-hveid="CBgQAA"><div jsname="jXK9ad" class="uMdZh tIxNaf rllt__borderless" jsaction="mouseover:UI3Kjd;mouseleave:Tx5Rb"><div class="VkpGBb"><div class="cXedhc"><a class="vwVdIc wzN8Ac rllt__link a-no-hover-decoration" jsname="kj0dLd" data-cid="6125476509096315320" jsaction="click:h5M12e;" role="link" tabindex="0" data-ved="2ahUKEwjGmqav77v_AhVPcvEDHe0ABA4Q1YkKegQIGBAB"><div><div class="rllt__details"><div class="dbg0pd" aria-level="3" role="heading"><span class="OSrXXb">Gustav Roasting Co.</span></div><div><span><span class="Y0A0hc"><span class="yi40Hd YrbPuc" aria-hidden="true">4.8</span><span class="z3HNkc" aria-label="Rated 4.8 out of 5," role="img"><span style="width:70px"></span></span><span class="RDApEe YrbPuc">(64)</span></span></span> · Coffee shop</div><div>Mehmet Akif Caddesi No:112</div><div class="dXnVAb"><span class="BI0Dve"><span><span aria-label="Takeaway">Takeaway</span></span><span class="if66xd">·</span></span><span class="BI0Dve"><span><span aria-label="No dine-in">No dine-in</span></span><span class="if66xd">·</span></span><span class="BI0Dve"><span><span aria-label="No delivery">No delivery</span></span></span></div></div></div></a><a class="uQ4NLd b9tNq wzN8Ac rllt__link a-no-hover-decoration" aria-hidden="true" tabindex="-1" jsname="kj0dLd" data-cid="6125476509096315320" jsaction="click:h5M12e;" role="link" data-ved="2ahUKEwjGmqav77v_AhVPcvEDHe0ABA4Q1YkKegQIGBAL"><g-img class="gTrj3e"><img id="pimg_1" src="https://lh5.googleusercontent.com/p/AF1QipMJBSZBgAw-jzVoRZWxuBUixiJJajyJ4ITrtslW=w167-h167-n-k-no" class="YQ4gaf zr758c wA1Bge" alt="" data-atf="4" data-frt="0" width="92" height="92"></g-img></a></div></div></div></div>',
+  '<a class="vwVdIc wzN8Ac rllt__link a-no-hover-decoration" jsname="kj0dLd" data-cid="8340045413518965442" jsaction="click:h5M12e;" role="link" tabindex="0" data-ved="2ahUKEwjVnbbfqr3_AhUEZ_EDHWpjALAQyTN6BAgQEAE"><div><div class="rllt__details"><div class="dbg0pd" aria-level="3" role="heading"><span class="OSrXXb">Fusion Kitchen</span></div><div><span><span class="Y0A0hc"><span class="yi40Hd YrbPuc" aria-hidden="true">4.6</span><span class="z3HNkc" aria-label="Rated 4.6 out of 5," role="img"><span style="width:62px"></span></span><span class="RDApEe YrbPuc">(30)</span></span></span> · Fast Food</div><div>Jordan, Hong Kong</div><div><span><b class="Z5bgrc">Takeout</b>: Now ⋅ Ends 10 pm</span></div></div></div></a>',
+]
+bearer_token = 'Huggingface Token or Private Server Key'
+server = 'Server URL' # Without the word Bearer
+
+time_start = Time.now
+results_and_time_taken = GoogleLocalResultsAiParser.parse_multiple(html_parts: html_parts, bearer_token: bearer_token, server: server, debug: true, no_cache: true)
+time_end = Time.now
+
+puts results_and_time_taken.map{|r| r[0]}
+puts "----"
+puts "Results"
+puts "Total Time Taken: #{time_end - time_start} seconds"
+puts "Maximum time taken in parallel requests: #{results_and_time_taken.map{|r| r[1]}.max}"
+
+# This is an example code to show you how you can debug the maximum time it takes to connect
+# to server in parallel requests and time it takes to parse. The model is forced to make a
+# prediction instead of serving cached results.

example_multiple_with_server.rb

@@ -0,0 +1,16 @@
+require './lib/google-local-results-ai-parser'
+
+html_parts = [
+  '<div jscontroller="AtSb" class="w7Dbne CR1S4b" data-record-click-time="false" id="tsuid_38" jsdata="zt2wNd;_;B40xrM WDO8Ff;_;B40xrY" jsaction="rcuQ6b:npT2md;e3EWke:kN9HDb" data-hveid="CBgQAA"><div jsname="jXK9ad" class="uMdZh tIxNaf rllt__borderless" jsaction="mouseover:UI3Kjd;mouseleave:Tx5Rb"><div class="VkpGBb"><div class="cXedhc"><a class="vwVdIc wzN8Ac rllt__link a-no-hover-decoration" jsname="kj0dLd" data-cid="6125476509096315320" jsaction="click:h5M12e;" role="link" tabindex="0" data-ved="2ahUKEwjGmqav77v_AhVPcvEDHe0ABA4Q1YkKegQIGBAB"><div><div class="rllt__details"><div class="dbg0pd" aria-level="3" role="heading"><span class="OSrXXb">Gustav Roasting Co.</span></div><div><span><span class="Y0A0hc"><span class="yi40Hd YrbPuc" aria-hidden="true">4.8</span><span class="z3HNkc" aria-label="Rated 4.8 out of 5," role="img"><span style="width:70px"></span></span><span class="RDApEe YrbPuc">(64)</span></span></span> · Coffee shop</div><div>Mehmet Akif Caddesi No:112</div><div class="dXnVAb"><span class="BI0Dve"><span><span aria-label="Takeaway">Takeaway</span></span><span class="if66xd">·</span></span><span class="BI0Dve"><span><span aria-label="No dine-in">No dine-in</span></span><span class="if66xd">·</span></span><span class="BI0Dve"><span><span aria-label="No delivery">No delivery</span></span></span></div></div></div></a><a class="uQ4NLd b9tNq wzN8Ac rllt__link a-no-hover-decoration" aria-hidden="true" tabindex="-1" jsname="kj0dLd" data-cid="6125476509096315320" jsaction="click:h5M12e;" role="link" data-ved="2ahUKEwjGmqav77v_AhVPcvEDHe0ABA4Q1YkKegQIGBAL"><g-img class="gTrj3e"><img id="pimg_1" src="https://lh5.googleusercontent.com/p/AF1QipMJBSZBgAw-jzVoRZWxuBUixiJJajyJ4ITrtslW=w167-h167-n-k-no" class="YQ4gaf zr758c wA1Bge" alt="" data-atf="4" data-frt="0" width="92" height="92"></g-img></a></div></div></div></div>',
+  '<a class="vwVdIc wzN8Ac rllt__link a-no-hover-decoration" jsname="kj0dLd" data-cid="8340045413518965442" jsaction="click:h5M12e;" role="link" tabindex="0" data-ved="2ahUKEwjVnbbfqr3_AhUEZ_EDHWpjALAQyTN6BAgQEAE"><div><div class="rllt__details"><div class="dbg0pd" aria-level="3" role="heading"><span class="OSrXXb">Fusion Kitchen</span></div><div><span><span class="Y0A0hc"><span class="yi40Hd YrbPuc" aria-hidden="true">4.6</span><span class="z3HNkc" aria-label="Rated 4.6 out of 5," role="img"><span style="width:62px"></span></span><span class="RDApEe YrbPuc">(30)</span></span></span> · Fast Food</div><div>Jordan, Hong Kong</div><div><span><b class="Z5bgrc">Takeout</b>: Now ⋅ Ends 10 pm</span></div></div></div></a>',
+]
+bearer_token = 'Huggingface Token or Private Server Key'
+server = 'Server URL' # Without the word Bearer
+
+
+results = GoogleLocalResultsAiParser.parse_multiple(html_parts: html_parts, bearer_token: bearer_token, server: server)
+
+puts results
+
+# This is an example code to show you how you can make multiple parsings
+# using your own hosted endpoint, or private server.