US 12,277,189 B2
System and method for URL fetching retry mechanism
Derry Shribman, Tel Aviv (IL); and Ofer Vilenski, Moshav Hadar Am (IL)
Assigned to BRIGHT DATA LTD., Netanya (IL)
Filed by BRIGHT DATA LTD., Netanya (IL)
Filed on Jan. 23, 2023, as Appl. No. 18/099,980.
Application 18/099,980 is a continuation of application No. 17/202,427, filed on Mar. 16, 2021, granted, now 11,593,446.
Application 17/202,427 is a continuation of application No. 16/939,077, filed on Jul. 27, 2020, granted, now 10,963,531, issued on Mar. 30, 2021.
Application 16/939,077 is a continuation of application No. 16/938,991, filed on Jul. 26, 2020.
Application 16/938,991 is a continuation of application No. PCT/IL2020/050194, filed on Feb. 21, 2020.
Claims priority of provisional application 62/809,847, filed on Feb. 25, 2019.
Claims priority of provisional application 62/855,036, filed on May 31, 2019.
Claims priority of provisional application 62/948,265, filed on Dec. 15, 2019.
Prior Publication US 2023/0161832 A1, May 25, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 16/909 (2019.01); G06F 9/54 (2006.01); G06F 16/955 (2019.01); H04L 1/00 (2006.01); H04L 67/01 (2022.01); H04L 67/02 (2022.01); H04L 67/146 (2022.01); H04L 67/568 (2022.01); H04L 69/40 (2022.01); H04L 61/59 (2022.01); H04L 67/56 (2022.01)
CPC G06F 16/9566 (2019.01) [G06F 9/547 (2013.01); G06F 16/909 (2019.01); G06F 16/955 (2019.01); G06F 16/9558 (2019.01); H04L 1/00 (2013.01); H04L 67/01 (2022.05); H04L 67/02 (2013.01); H04L 67/146 (2013.01); H04L 67/568 (2022.05); H04L 69/40 (2013.01); H04L 61/59 (2022.05); H04L 67/56 (2022.05)] 34 Claims
OG exemplary drawing
 
1. A method for use with a content that comprises a web-page or a part thereof and that is stored only in a web server and is identified by a Uniform Resource Locator (URL), for use with a first and second lists of Internet Protocol (IP) addresses, the method comprising:
identifying, by a first device, the URL;
selecting, by the first device, a first IP address from the first list;
sending, by the first device to the web server over the Internet, a first Hypertext Transfer Protocol (HTTP) request that comprises the URL, using the selected first IP address as a source address;
receiving, by the first device over the Internet, a first response in response to the sent first HTTP request;
determining whether the first response is a proper response that comprises the content; and
responsive to determining that the first response is not a proper response:
selecting, by the first device, a second IP address from the second list;
sending, by the first device to the web server over the Internet, a second Hypertext Transfer Protocol (HTTP) request that comprises the URL, using the selected second IP address as a source address;
receiving, by the first device over the Internet, a second response in response to the sent second HTTP request;
determining whether the second response is a proper response that comprises the content;
responsive to determining that the second response is a proper response, using the content received in the second response; and
responsive to determining that the second response is not a proper response, performing, by the first device, a first action.