US 12,250,090 B2
System and method for improving content fetching by selecting tunnel devices
Derry Shribman, Tel Aviv (IL); and Ofer Vilenski, Moshav Hadar Am, IL (US)
Assigned to BRIGHT DATA LTD., Netanya (IL)
Filed by BRIGHT DATA LTD., Netanya (IL)
Filed on Apr. 2, 2024, as Appl. No. 18/624,563.
Application 18/624,563 is a continuation of application No. 18/209,863, filed on Jun. 14, 2023, granted, now 11,956,094.
Application 18/209,863 is a continuation of application No. 17/209,867, filed on Mar. 23, 2021, granted, now 11,729,013, issued on Aug. 15, 2023.
Application 17/209,867 is a continuation of application No. 17/062,717, filed on Oct. 5, 2020, granted, now 10,985,934, issued on Apr. 20, 2021.
Application 17/062,717 is a continuation of application No. 17/062,592, filed on Oct. 4, 2020, granted, now 11,729,012, issued on Aug. 15, 2023.
Application 17/062,592 is a continuation of application No. 16/481,470, granted, now 10,880,266, issued on Dec. 29, 2020, previously published as PCT/IL2018/050910, filed on Aug. 16, 2018.
Claims priority of provisional application 62/684,211, filed on Jun. 13, 2018.
Claims priority of provisional application 62/624,208, filed on Jan. 31, 2018.
Claims priority of provisional application 62/563,157, filed on Sep. 26, 2017.
Claims priority of provisional application 62/550,834, filed on Aug. 28, 2017.
Prior Publication US 2024/0275628 A1, Aug. 15, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 7/58 (2006.01); G06F 8/71 (2018.01); G06F 9/455 (2018.01); G06F 9/48 (2006.01); G06F 16/955 (2019.01); H04L 9/40 (2022.01); H04L 12/28 (2006.01); H04L 12/46 (2006.01); H04L 47/283 (2022.01); H04L 61/256 (2022.01); H04L 61/2575 (2022.01); H04L 61/2585 (2022.01); H04L 61/2589 (2022.01); H04L 61/2592 (2022.01); H04L 61/4511 (2022.01); H04L 61/5007 (2022.01); H04L 67/01 (2022.01); H04L 67/02 (2022.01); H04L 67/025 (2022.01); H04L 67/133 (2022.01); H04L 67/141 (2022.01); H04L 67/142 (2022.01); H04L 67/288 (2022.01); H04L 67/2885 (2022.01); H04L 67/56 (2022.01); H04L 67/563 (2022.01); H04L 67/568 (2022.01); H04L 67/5681 (2022.01); H04L 67/63 (2022.01); H04L 69/16 (2022.01); H04L 69/167 (2022.01); H04L 69/168 (2022.01); H04W 4/80 (2018.01); H04W 48/18 (2009.01); H04W 84/10 (2009.01); H04W 84/18 (2009.01); H04L 101/69 (2022.01)
CPC H04L 12/2803 (2013.01) [G06F 7/58 (2013.01); G06F 7/588 (2013.01); G06F 8/71 (2013.01); G06F 9/45545 (2013.01); G06F 9/4555 (2013.01); G06F 9/4881 (2013.01); G06F 16/955 (2019.01); H04L 12/2856 (2013.01); H04L 12/4633 (2013.01); H04L 47/283 (2013.01); H04L 61/256 (2013.01); H04L 61/2575 (2013.01); H04L 61/2585 (2013.01); H04L 61/2589 (2013.01); H04L 61/2592 (2013.01); H04L 61/4511 (2022.05); H04L 61/5007 (2022.05); H04L 63/0272 (2013.01); H04L 63/0281 (2013.01); H04L 63/029 (2013.01); H04L 63/164 (2013.01); H04L 67/01 (2022.05); H04L 67/02 (2013.01); H04L 67/025 (2013.01); H04L 67/133 (2022.05); H04L 67/141 (2013.01); H04L 67/142 (2013.01); H04L 67/288 (2013.01); H04L 67/2885 (2013.01); H04L 67/56 (2022.05); H04L 67/563 (2022.05); H04L 67/568 (2022.05); H04L 67/5681 (2022.05); H04L 67/63 (2022.05); H04L 69/16 (2013.01); H04L 69/162 (2013.01); H04L 69/167 (2013.01); H04L 69/168 (2013.01); H04W 4/80 (2018.02); H04W 48/18 (2013.01); H04W 84/10 (2013.01); H04W 84/18 (2013.01); G06F 9/45558 (2013.01); G06F 2009/45591 (2013.01); G06F 2009/45595 (2013.01); H04L 2101/69 (2022.05)] 32 Claims
OG exemplary drawing
 
1. A method for operating a database that associates a respective value indicative of performance and availability of a group of servers, each of the servers in the group stores a respective distinct list of Internet Protocol (IP) addresses, the method comprising:
identifying a first content that comprises a first web page or a part thereof;
selecting, from the group of servers, based on the associated value in the database and on load balancing, a first server that is operated by a first proxy provider and that stores a first list of IP addresses;
selecting, a first IP address from the first list, by using a random selection, a sequential selection, or a cycling selection;
sending, to a first client device that is addressed by the selected first IP address over the Internet, a first request for the first content;
receiving, from the first client device over the Internet, in response to the sending of the first request, the first content;
measuring, a first delay associated with the receiving of the first content;
updating, in the database, a value associated with the first server, in response to, or based on, the measured first delay;
identifying a second content that comprises a second web page or a part thereof;
selecting, from the group of servers, based on the associated value in the database and on load balancing, a second server that stores a second list of IP addresses, and that is operated by a second proxy provider that is operated by a second proxy provider that is distinct from the first proxy provider;
selecting, a second IP address from the second list, by using a random selection, a sequential selection, or a cycling selection;
sending, to a second client device that is addressed by the selected second IP address over the Internet, a second request for the second content;
receiving, from the second client device over the Internet, in response to the sending of the second request, the second content;
measuring, a second delay associated with the receiving of the second content; and
updating, in the database, a value associated with the second server, in response to, or based on, the measured second delay,
periodically reconfiguring proxy provider's utilization threshold according to the performance and availability information of a server from the group wherein the performance and availability information of the first or second server comprises a respective server identification, a respective server status, a respective server activity, a size of data obtained through the respective server, or any combination thereof,
wherein the measuring of the first or second delay further comprises measuring or estimating of Round-Trip delay Time (RTT), Bandwidth (BW), or Packet Loop Ratio (PLR).