Application Note: Video – ANV073
Exploring Brivo Eagle Eye Video Search Capabilities: Objects, Query, Filtering
2024-11-26 Revision 1.0
This Application Note is intended for all end users of Brivo Eagle Eye Video who have a need to search historical video and want to better understand how Smart Video Search works, or to find something or someone across multiple cameras or locations. A working knowledge of Brivo Eagle Eye Video is recommended before learning more about Smart Video Search.
Brivo Eagle Eye Smart Video Search is an AI-powered solution that allows users to search for people, vehicles, or objects using natural language, similar to how people search Google or other search engines. Read AN052 Leveraging Smart Video Search for Efficient Investigations for more information.
Brivo Eagle Eye Smart Video Search supports a set of objects and a list of attributes that describe the objects. A user can search for these objects and use their attributes to filter the search. Below is a list of currently supported objects and their associated attributes:
Object: Person
Attributes: Gender (male/female), Upper Body Clothing Color, Lower Body Clothing Color
Object: Vehicle
Attributes: Color, Make, Body Type (car, bus, van, truck)
Other Objects: Handbag, Backpack, Suitcase, Bicycle
Note: Color searches are not currently supported for handbags, backpacks, suitcases, or bicycles.
When a user clicks on the Video Search bar, a dropdown will appear with a history of recent searches and suggestions of objects that can be searched.
The historical searches are represented by the clock icon and each of the recommended search objects have their own icons.
When a user clicks on a suggested search item, Eagle Eye runs a video search for the item and displays a set of results related to the search. These search results are from the past 24 hours by default, and are grouped by camera. The example below shows the results for a search for the suggested term “person.” Three dropdown items appear, namely Camera, Time, and Person. The third dropdown represents the searched item.
Users of the Professional and Enterprise Editions will see two options: Cameras and Locations. Selecting “Cameras” will group the search results by camera. Selecting “Locations” groups the search results instead by location.
The user can select any of the sub-filters in this dropdown to further filter the results by Locations, Cameras, Tags or ROI (Region of Interest). The results will be updated based on these selected filters. The user can also select multiple options in any of these filters.
The dropdown labeled Time can be used in the following ways:
Date: This allows the user to select the date for the search. Only the dates underlined in blue can be selected, as Video Search is available only for the past 7 days.
Start or End Time: Users can select the start or end time by clicking on the time bar; the selection period will move in response.
Selection period: Users can select either a 1-hour, 4-hour, 12-hour or 24-hour search window by clicking on the preferred option.
Users can select one or more object attributes to narrow down the search. In the below example, the search attributes describe a person with both black upper-body clothing and black lower-body clothing.
This can also be typed out in the search bar. Besides color, users can also enter a wide range of related clothing descriptors, such as shirt, top, t-shirt, pants, and jeans. The example below is a result triggered by a search for a person in blue upperwear and black lower.
Users can click on the delete button (at the left side of the submenu dropdowns), and start a fresh search.
The best way to use Eagle Eye Smart Video Search is to construct the query by using the suggestions and drop-down filters in the Eagle Eye Cloud VMS. This ensures there are no spelling mistakes or syntax errors. Given below are the best practices for getting the correct results:
Always start with the object or search instead of the attributes. For example, "person in a red shirt" is a better search query than "red shirt person."
To use more than one attribute to describe the object, use the word “and.” For example, a person in a red shirt and blue pants.
In the case of a vehicle, describe the type of vehicle — for example, "black car" or "blue truck."
To search for more than one object in the same query, use the word “with.” For example, "person in red shirt with backpack." If you want the results with both a person and car, the query should be “person with car.” Note: It should not be “person in a car.” The query “person with car” will search for results including both a person and a car.
Alternatively, you can also type it with a semicolon as a separator, for example, “person; car.”
If the user were to search for “person in car,” Video Search would drop the word "car" because car is not an attribute to describe a person.
Think of the Brivo Eagle Eye video search system a bit like using Google: when you search for something specific, like “best Italian restaurant,” the first page doesn’t always show exactly what you want. It might list some places that aren't a perfect match but that are relevant based on keywords or location. Similarly, in our video search, when you look for a “red shirt,” the system aims to include all relevant matches, which sometimes means a mostly red shirt with a small blue logo or a predominantly blue shirt with a red accent might appear.
Just as Google balances between relevance and variety to give you a fuller set of options, our video search system uses primary and secondary colors to broaden search results and avoid missing potential matches—though it may mean a bit of sorting on the user’s end to get the most precise result.
When a user searches for an object, Brivo Eagle Eye AI looks for that object in the recorded video stream and displays results for every time the searched object was found. Hence, the results page will have multiple images from each of the selected cameras across the selected time period.
The user can then navigate through all the results from any of the cameras by clicking on arrows. The arrow to the left shows the previous result, and the arrow to the right takes the user to the next result.
The user can look at a result in greater detail by clicking on the result. This opens the result in a separate window that shows all the search-related objects in the image in bounding boxes along with a description in the bottom.
Hovering over any of the descriptions highlights the related object in the image. If a user clicks on the description, a new search is triggered with the selected key phrase as the search query.
Smart Video Search is a tool meant to help find an object of interest in the video, thereby reducing the need to manually watch and search through the video. Since the objective is to help the end user find possible objects of interest, the Video Search AI uses an expanded definition of attributes to provide more search results to increase the chance of finding objects of interest.
Use of primary and secondary colors: When a color attribute is used for search, Video Search provides results with the color selected by the user, but also with results that are similar to the mentioned color. This is done intentionally to show the likely object of interest by displaying results that may be of interest to the user. Below is one of the results for a search for a “person in blue upperwear.”
The same result would appear for a search for a “person in gray upperwear” as well.
Note: This loose matching is not an error or a bug, but an intentional approach to increase the chances of finding what the user is looking for, both because colors can appear different under different lighting conditions and because color terms themselves can be ambiguous
Several factors can reduce the accuracy of video search results:
Colored indoor lighting: Strongly colored light sources, and multiple sources with different colors can result in heavy distortion in objects' perceived color.
Reflective floors: Reflective flooring can cause distortion in colors.
Odd angles: Surveillance cameras set up at extreme angles can make it hard to spot or distinguish upperwear and lowerwear colors.