CPC G06F 16/245 (2019.01) [G06F 16/285 (2019.01); G06F 16/951 (2019.01); H04L 51/216 (2022.05); H04L 51/226 (2022.05); H04L 51/52 (2022.05); G06Q 50/01 (2013.01)] | 14 Claims |
1. A method for setting a schedule of a crawl of a content from a web content page, the method comprising:
parsing, by a processor, the content into a first portion and a second portion, the first portion being associated with a core textual content in the web content page, the second portion being associated with a content related to the core textual content;
determining, by the processor, whether the second portion of the web content page is a first content modification type or a second content modification type, wherein a content modification type indicates whether new content exists in the second portion of the web content page;
causing the processor, in response to a determination that the web content page is the first content modification type, to increase a time duration for a next fetch of the core textual content;
setting, by the processor, the schedule according to a content modification type of the web content page, the content modification type being one of the first content modification type or the second content modification type; and
performing, by the processor, the next fetch of the core textual content according to the schedule.
|