Overview

The summary endpoint uses Claravine AI to classify content (video and image) along with associated metadata when available. IAB Categories and GARM Safety are returned by default. Multiple “add-on” modules are also available, which provide additional data elements to include (detailed in the Parameters section).

IAB Classification

The content is classified into the IAB Categories, and when appropriate, into IAB subcategories. The data are expressed in a multi-label classification format with multi-tier predictions collapsed by key, when appropriate.

Content Safety

The Safety model implements the GARM guidelines for content safety. The safety key provides an assessment of the visual contents in conjunction with metadata.

The risk key is an overall safety estimate of video content into High, Medium, and Low. We recommend using the Risk key as the determinant for triage.

In addition to the risk characterization, we also provide the specific type(s) of the offending content. The GARM categories we cover are Terrorism, Death Injury or Military Conflict, Adult & Explicit Sexual Content, Hate Speech, Illegal Drugs, Crime & Harmful Acts, Sensitive Social Issues, Spam & Harmful Content, Obscenity & Profanity, and Arms & Ammunition.

Note: The GARM class itself and risk are orthogonal to each other and provide a consistent, transparent, and clear way of flagging unsafe segments within the content.

Scenes

The scenes collection is keyed by the time offset with 2 sub dictionaries:

context: the objects, and categories of the objects (based on IAB taxonomy)
safety: It shows a risk score and in addition, if risk is “Medium” or ”High”, the key includes the offending GARM category.

Depending on the selected add-ons, corresponding items are included (e.g. object, place, activity, etc)

Note: Scene-level safety is aggregated into a video level safety assessment in the Summary key.

Response

Upon making a successful request you should receive an acknowledgment callback in the format shown.

In the event of an error, the callback would include an error message with the appropriate HTTP error code (e.g., 4xx or 5xx). A 2xx message indicates that the request was accepted successfully.