Calvin Huang

Scale relies on Mode Analytics for internal SQL and Python based reporting and analytics, helping power our operations, insights and business decisions. Many of these reports we set to run on a schedule, so that we can always go to a report and see data that’s no more than fifteen minutes to an hour out of date. However, Mode doesn’t natively support a method of notifying people when something needs attention, e.g. when our margins on a project take a sharp dive or many new tasks are suddenly created. We’ve set up a system to utilize these reports to easily alert us when something’s unusual, so that someone can take action.<h2>Alert generation</h2> <img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Ff0583e4d10994410bb5db605266bf655" alt="Scale and Slack Dashboards"> The flow for alerting goes as follows: when a report is run (either manually or on a schedule), it sends a webhook to an AWS Lambda function, which then pulls the report data from the Mode API, and retrieves a list of alert definitions from our database. It then looks through the alert definitions and sees if any are applicable to the current report being run, and if so, creates an alert for each row in the report that satisfies conditions set out in the matching alert definition. These alerts are then sent to a Slack webhook, which generates an alert in a channel specified in the alert definition.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/2b8f800f-0c2f-4a52-53d5-d2a3205d8100/public" alt="Slack Dashboard"> <h2>Defining alerts</h2> <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/8a14c54f-48ce-48b0-0d62-dbe006402700/public" alt="Alerts Dashboard"> Alerts are defined in our database, and are easily viewable/creatable from our corp dashboard.Alerts are simply defined with a handful of fields: <ul><li>Report URL: the URL of the Mode report that this alert can</li><li> fire for.</li><li> </li><li>Alert Title</li><li>Alert Condition: Python expression that is evaluated with</li><li> every row of a report to determine if that row is alert-worthy.</li><li> </li><li>Filter-max Column: name of a column to only take rows where</li><li> the value of that column is maximal. Useful for reports with time-series</li><li> data, usually you only want to alert on rows with the most recent data.</li><li> </li><li>Alert Text Format String: Python format string, used to</li><li> generate the message text in the Slack alert.</li><li> </li><li>Color: sidebar color of the alert message</li><li>Alert Channel: Slack channel to post to</li><li>Enabled</li><li> </li></ul><img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2F54c463b0ae5740d5af829e3d57c46a96" alt="Report Dashboard"> The combination of the easy-to-use dashboards, and the simple definitions of the alerts, makes creating alerts based on reports very easy to non-technical people—all that’s required to make fully-featured custom alerts is very basic Python knowledge—essentially basic math operations—and SQL if one is creating their own reports from scratch.

Scale relies on Mode Analytics for internal SQL and Python based reporting and analytics, helping power our operations, insights and business decisions.

How to Trigger Alerts With Mode Analytics - Scale

How We Made It Easy for Anyone at Our Company to Trigger Alerts

Here at Scale, one of the image annotation services we offer is&nbsp;<a href="https://scale.com/3d-cuboids" rel="noopener noreferrer" target="_blank">Cuboid Annotation</a>, which annotates your two-dimensional images with projections of cuboids enclosing objects such as cars, trucks, pedestrians, traffic cones, you name it. With some additional information, we can turn those two-dimensional box annotations into full, three-dimensional boxes, with height, width, depth, rotation, and relative positioning info. <h2>Manual cuboid annotation</h2> With manual cuboid adjustment, scalers simply draw a 2D box representing one side of the cuboid and an additional side of the cuboid: <img src="https://drive.tiny.cloud/1/0lbhu31lrngl0y5eo8e6p0jckqgeikc94mnkaig43yuahupv/80ca67fd-00d1-4e55-b7a0-e1d22c826c88" alt="cuboid selection" height="186" width="240">ordinary cuboid This often does not result in a "true" cuboid as it is mathematically imprecise. The front face of a "true" cuboid will likely not be a perfect 90 degree rectangle, especially if it isn't facing the camera head-on. Furthermore, edges of a cuboid parallel to the ground should converge to the horizon, whereas the top/bottom edges of the right side of the above annotation are parallel. In our task response, we provide the image coordinates of the points and edges: <pre class="ql-syntax" spellcheck="false">{
 "edges": [
 {"description": "face-top", "x1": 236, "y1": 308, "x2": 328, "y2": 308, "type": "edge"},
 {"description": "face-right", ...},
 {"description": "face-bottom", ...},
 {"description": "face-left", ...},
 {"description": "side-bottom", ...},
 {"description": "side-top", ...}
 ],
 "vertices": [
 {"description": "face-topleft", "x": 236, "y": 308, "type": "vertex"},
 {"description": "face-bottomleft", ...},
 {"description": "face-topright", ...},
 {"description": "face-bottomright", ...},
 {"description": "side-topcorner", ...},
 {"description": "side-bottomcorner", ...}
 ],
 "label": "car"
}
</pre><h2>Adjusted cuboid</h2> Given the cuboid annotation above, and some additional information (namely, camera parameters and orientation), we automatically generate a more accurate annotation: The front face is no longer perfectly rectangular, but trapezoidal with the left edge slightly smaller, better reflecting the relative orientation of the car to the camera. The top/bottom edges of the right side now converge to a point on the horizon, as they should. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/767c2267-2c47-46e3-91ab-be1281020400/public" alt="adjusted cuboid selection">adjusted cuboid After cuboid adjustment, we add two new fields&nbsp;<code>points_3d</code> and <code>points_2d</code> to the original annotation: <pre class="ql-syntax" spellcheck="false">{
 "edges": {...},
 "vertices": {...},
 "label": "car",
 "points_3d": [
 {"x": 14.903, "y": 4.117, "z": -0.0928},
 ... // 7 more points
 ],
 "points_2d": [
 {"x": 235.39, "y": 308.05},
 ... // 7 more points
 ]
}
</pre>which represent the 3D coordinates of the vertices1 of the cuboid in space (relative to the position of the camera, in meters) and the pixel coordinates of the resulting 2D projection. From these eight points in 3D-space it's easy to infer the position, dimensions, and orientation of the resulting cuboid. With these attributes, one can train models such as <a href="https://arxiv.org/pdf/1611.10010.pdf" rel="noopener noreferrer" target="_blank">Magic Leap's Deep Cuboid Detection</a>, which predicts 3d coordinates of cuboid-like objects from a single image. In this way, one can create a system to identify the locations of cars in the world, using only camera images and annotations generated by Scale. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/e66af6d5-7c8c-4568-a6cd-846a247edd00/public" alt="cuboid annotations">Deep Cuboid Detection Pipeline <h2>Further examples</h2> Initial AnnotationAfter Adjustment <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/2dd29cfd-8211-41d1-e0a0-258189a9db00/public"><img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/f2006f44-0454-466a-f24e-8a9076476c00/public"> <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/5cca4474-7fce-4132-3e36-fa409ae90200/public"><img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/c6463b92-2b20-459e-f506-e2aa665efc00/public"> <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/50106285-eed1-4ef8-0a8a-f3f263730600/public"><img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/68665344-da48-47a0-7c9e-eae355031100/public"> Because it's impossible to visually distinguish between one object, and another object twice the size and twice as far away, it's only possible to calculate 3D coordinates up to a scaling factor (i.e. you can multiply the points_3d of the resulting cuboid by an arbitrary value, and it would be just as valid). With additional data, such as depth data from LIDAR or from stereo imaging, one knows the exact distance from the front face of the cuboid and can scale the points accordingly. Alternately, if the elevation of the camera is provided, we scale the resulting cuboids so that the bottom face of each cuboid touches the ground.

Calvin Huang

How We Made It Easy for Anyone at Our Company to Trigger Alerts

Adding a Dimension: Annotating 3d Objects with 2d Data