| CPC G06N 20/20 (2019.01) [G06F 40/279 (2020.01); G06F 40/30 (2020.01); G06V 20/41 (2022.01)] | 10 Claims |
|
1. A computer system comprising:
a first computer configured to execute a machine learning content moderation component, configured to:
receive input data representative of a media post or message made on an online platform;
receive a behaviour profile selected from a plurality of behaviour profiles;
analyse the input data using a plurality of machine learning models, the plurality of machine learning models including a key word identification model to detect offensive words and a toxicity model to detect toxicity;
store a plurality of combination logics, each combination logic comprising a plurality of predefined rules for combining outputs of the plurality of machine learning models, a first rule of the plurality of rules comprising a threshold applied to a first output of the outputs, each combination logic corresponding to a respective one of the plurality of behaviour profiles;
select, based on the received behaviour profile, a corresponding combination logic of the plurality of combination logics, and
combine outputs of the plurality of machine learning models using the selected combination logic to generate a moderation result indicating whether the media post or message contains offensive content,
wherein the machine learning content moderation component comprises a stateless application programming interface (API) configured to receive the input data and provide the moderation result;
wherein the computer system further comprises a second computer configured to execute a content analysis component, the content analysis component configured to:
obtain a media post or message from an online platform using a client API provided by the online platform;
determine the behaviour profile that is associated with an account holder having an account on the online platform, wherein the media post or message is intended for display to the account holder, wherein each behaviour profile of the plurality of behaviour profiles represents a plurality of account holders having a same set of content viewing preferences;
transmit the input data and determined behaviour profile to the machine learning content moderation component on the first computer via the stateless API;
receive the moderation result from the machine learning content moderation component on the first computer; and
in response to the moderation result indicating that the media post or message comprises offensive content, communicate with the online platform via the client API to delete or hide the media post or message,
wherein the first computer is not configured to execute the content analysis component and the second computer is not configured to execute the machine learning content moderation component.
|