US 11,729,013 B2
System and method for improving content fetching by selecting tunnel devices
Derry Shribman, Tel Aviv (IL); and Ofer Vilenski, Moshav Hadar Am (IL)
Assigned to BRIGHT DATA LTD., Netanya (IL)
Filed by BRIGHT DATA LTD., Netanya (IL)
Filed on Mar. 23, 2021, as Appl. No. 17/209,867.
Application 17/209,867 is a continuation of application No. 17/062,717, filed on Oct. 5, 2020, granted, now 10,985,934.
Application 17/062,717 is a continuation of application No. 17/062,592, filed on Oct. 4, 2020.
Application 17/062,592 is a continuation of application No. 16/481,470, granted, now 10,880,266, issued on Dec. 29, 2020, previously published as PCT/IL2018/050910, filed on Aug. 16, 2018.
Claims priority of provisional application 62/684,211, filed on Jun. 13, 2018.
Claims priority of provisional application 62/624,208, filed on Jan. 31, 2018.
Claims priority of provisional application 62/563,157, filed on Sep. 26, 2017.
Claims priority of provisional application 62/550,834, filed on Aug. 28, 2017.
Prior Publication US 2021/0234721 A1, Jul. 29, 2021
This patent is subject to a terminal disclaimer.
Int. Cl. H04L 9/40 (2022.01); H04L 61/5007 (2022.01); H04L 67/2885 (2022.01); H04L 12/28 (2006.01); H04L 61/256 (2022.01); H04L 67/025 (2022.01); H04L 69/16 (2022.01); G06F 9/48 (2006.01); H04L 61/2592 (2022.01); H04L 67/02 (2022.01); H04L 67/288 (2022.01); G06F 7/58 (2006.01); H04L 61/2575 (2022.01); H04L 61/2585 (2022.01); H04L 61/2589 (2022.01); H04L 12/46 (2006.01); H04L 67/141 (2022.01); H04L 67/142 (2022.01); H04W 4/80 (2018.01); G06F 16/955 (2019.01); G06F 8/71 (2018.01); G06F 9/455 (2018.01); H04L 47/283 (2022.01); H04L 69/167 (2022.01); H04L 69/168 (2022.01); H04W 48/18 (2009.01); H04W 84/10 (2009.01); H04W 84/18 (2009.01); H04L 61/4511 (2022.01); H04L 67/01 (2022.01); H04L 67/56 (2022.01); H04L 67/63 (2022.01); H04L 67/133 (2022.01); H04L 67/563 (2022.01); H04L 67/5681 (2022.01); H04L 67/568 (2022.01); H04L 101/69 (2022.01)
CPC H04L 12/2803 (2013.01) [G06F 7/58 (2013.01); G06F 7/588 (2013.01); G06F 8/71 (2013.01); G06F 9/4555 (2013.01); G06F 9/45545 (2013.01); G06F 9/4881 (2013.01); G06F 16/955 (2019.01); H04L 12/2856 (2013.01); H04L 12/4633 (2013.01); H04L 47/283 (2013.01); H04L 61/256 (2013.01); H04L 61/2575 (2013.01); H04L 61/2585 (2013.01); H04L 61/2589 (2013.01); H04L 61/2592 (2013.01); H04L 61/4511 (2022.05); H04L 61/5007 (2022.05); H04L 63/029 (2013.01); H04L 63/0272 (2013.01); H04L 63/0281 (2013.01); H04L 63/164 (2013.01); H04L 67/01 (2022.05); H04L 67/02 (2013.01); H04L 67/025 (2013.01); H04L 67/133 (2022.05); H04L 67/141 (2013.01); H04L 67/142 (2013.01); H04L 67/288 (2013.01); H04L 67/2885 (2013.01); H04L 67/56 (2022.05); H04L 67/563 (2022.05); H04L 67/568 (2022.05); H04L 67/5681 (2022.05); H04L 67/63 (2022.05); H04L 69/16 (2013.01); H04L 69/162 (2013.01); H04L 69/167 (2013.01); H04L 69/168 (2013.01); H04W 4/80 (2018.02); H04W 48/18 (2013.01); H04W 84/10 (2013.01); H04W 84/18 (2013.01); G06F 9/45558 (2013.01); G06F 2009/45591 (2013.01); G06F 2009/45595 (2013.01); H04L 2101/69 (2022.05)] 28 Claims
OG exemplary drawing
 
1. A method performed by proxy servers from a group of proxy servers, utilizing a database that associates a respective service level value to each of the proxy servers in the group, the method comprising:
identifying, by a client device, a first proxy server, that is operated by a first proxy provider, from the group that stores a first list of IP addresses;
establishing a first connection to a first web server through the first proxy server in response to the identifying of the first proxy server;
selecting a first IP address from the first list;
obtaining a first content from a first web server using the first IP address;
measuring a first error or a first delay associated with the obtaining of the first content from the first proxy server;
updating the service level value of the first proxy server based on the respective measured first error or first delay;
identifying, by the client device, a second proxy server, that is operated by a second proxy provider that is distinct from the first proxy provider, from the group that stores a second list of IP addresses;
establishing a second connection to a second web server through the second proxy server in response to the identifying of the second proxy server;
selecting a second IP address from the second list; obtaining a second content from a second web server using the second IP address;
measuring a second error or a second delay associated with the obtaining of the second content from the second proxy server;
updating the service level value of the second proxy server based on the respective measured second error or second delay; and
calculating and configuring utilization thresholds by evaluating at least one of performance and availability information of the first proxy server and the performance and availability information of the second proxy server,
wherein the first web server comprises, or consists of, a first web server for serving the content accessible through Hypertext Transfer Protocol (HTTP) or HTTP Secure (HTTPS) protocols, and wherein the second web server comprises, or consists of, a second web server for serving the content accessible through HTTP or HTTPS protocols,
wherein each of IP addresses in the first and second lists is associated with a respective client device that is configurable to serve as a respective proxy device for obtaining a respective content from a respective web server, and
wherein the selecting of the first IP address or the selecting of the second IP address uses, or is based on, random selection, sequential or cycling selection, a criterion associated with a value of an attribute, or any combination thereof.