Example: Model Annotations Service

This tutorial demonstrates creating and deploying a service that pre-annotates an items before manual annotation work is performed, as part of active-learning process.

Service Code

Your code can perform any action you need toe execute as part of pre-annotating items for example:

  • Access a remote server/API to retrieve annotations
  • Run your algorithm or ML model as a service in Dataloop FaaS

In this example we use a simple face detection algorithm that uses Cv2 and Caffe model

Copy
Copied
import numpy as np
import os
import cv2
import dtlpy as dl
class ServiceRunner:
    def __init__(self,
                 model_filename: str,
                 prototxt_filename: str,
                 min_confidence: float):
        prototxt = os.path.join(os.getcwd(), prototxt_filename)
        weights = os.path.join(os.getcwd(), model_filename)
        print("[INFO] loading model...")
        self.net = cv2.dnn.readNetFromCaffe(prototxt, weights)
        self.min_confidence = min_confidence
    def detect(self, item: dl.Item):
        print("[INFO] downloading image...")
        filename = item.download()
        try:
            # load the input image and construct an input blob for the image
            # by resizing to a fixed 300x300 pixels and then normalizing it
            print("[INFO] opening image...")
            image = cv2.imread(filename)
            (h, w) = image.shape[:2]
            blob = cv2.dnn.blobFromImage(cv2.resize(image,
                                                    (300, 300)), 1.0,
                                         (300, 300),
                                         (104.0, 177.0, 123.0))
            # pass the blob through the network and obtain the detections and
            # predictions
            print("[INFO] computing object detections...")
            self.net.setInput(blob)
            detections = self.net.forward()
            # create annotation builder to add annotations to item
            print("[INFO] uploading annotations...")
            builder = item.annotations.builder()
            # loop over the detections
            for i in range(0, detections.shape[2]):
                # extract the confidence (i.e., probability) associated with the
                # prediction
                confidence = detections[0, 0, i, 2]
                # filter out weak detections by ensuring the `confidence` is
                # greater than the minimum confidence
                if confidence > self.min_confidence:
                    # compute the (x, y)-coordinates of the bounding box for the
                    # object
                    box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
                    (startX, startY, endX, endY) = box.astype("int")
                    # draw the bounding box of the face along with the associated
                    # probability
                    builder.add(
                        annotation_definition=dl.Box(
                            top=startY,
                            left=startX,
                            right=endX,
                            bottom=endY,
                            label='person'
                        ),
                        model_info={
                            'name': 'Caffe',
                            'confidence': confidence
                        }
                    )
                    # upload annotations
            builder.upload()
            print("[INFO] Done!")
        finally:
            os.remove(filename)

Define the module

In this example, we load the model in the init method, which runs only once at deployment time, saving us time bu not loading on each execution/
This can inputs are attributes that we want the service to include for its entire lifetime. In this case, it's the model and weights files we want the service to use and the confidence limit for accepting detections.

pythonpython
Copy
Copied
module = dl.PackageModule(
    init_inputs=[
        dl.FunctionIO(name='model_filename', type=dl.PackageInputType.STRING),
        dl.FunctionIO(name='prototxt_filename', type=dl.PackageInputType.STRING),
        dl.FunctionIO(name='min_confidence', type=dl.PackageInputType.FLOAT)
    ],
    functions=[
        dl.PackageFunction(
            name='detect',
            description='OpenCV face detection using Caffe model',
            inputs=[
                dl.FunctionIO(name='item', type=dl.PackageInputType.ITEM)
            ]
        )
    ]
)
Copy
Copied
numpy == 1.18
opencv - python == 3.4

Model and weights files

The function uses 2 files containing the model and its weights for inferencing detections. We need to have these files at the same folder as the entry point.
To get these files please download them here.
https://storage.googleapis.com/dtlpy/model_assets/faas-tutorial/model_weights.zip

Package Requirements

Our package's codebase uses 2 Python libraries that are not standard ones. Therefore, we need to make sure they are pre-installed before running the entry point. One way to do so is to use a custom Docker Image (information on this process can be found here. The other way is to add a requirements.txt file to the package codebase. To do so, simply add the following requirements.txt file in the same folder of the entry point (main.py):
https://dataloop.ai/docs/service-runtime#customimage

Copy
Copied
package = project.packages.push(
    src_path='<path to folder containing the codebase>',
    package_name='face-detector',
    modules=[module]
)

Push the Package

Make sure you have the following files in one directory:

  • main.py
  • requirements.txt
  • res10 300x300 ssd iter 140000.caffemodel
  • deploy.prototxt.txt

Run this to push your package:

Copy
Copied
service = package.deploy(
    service_name='face-detector',
    init_input=[
        dl.FunctionIO(name='model_filename',
                      type=dl.PackageInputType.STRING,
                      value='res10_300x300_ssd_iter_140000.caffemodel'),
        dl.FunctionIO(name='prototxt_filename',
                      type=dl.PackageInputType.STRING,
                      value='deploy.prototxt.txt'),
        dl.FunctionIO(name='min_confidence',
                      type=dl.PackageInputType.FLOAT,
                      value=0.5)
    ],
    runtime=dl.KubernetesRuntime(concurrency=1)
    # The runtime argument Concurrency=1 means that only one execution can run at a time (no parallel executions).
)

Deploy The Service

The package is now ready to be deployed as a service in the Dataloop Platform.
Whenever executed, your package will run as a service on default instance type. Review the service configuration to configure it to your needs, for example

  • Change instance-type to use stronger instances with more memory, CPU and GPU
  • Increase auto-scaling to handle larger loads
  • Increase timeouts to allow longer execution time
Copy
Copied
filters = dl.Filters(resource=dl.FiltersResource.ITEM)
filters.add(field='metadata.system.mimetype', values='image*')
trigger = service.triggers.create(
    name='face-detector',
    function_name="detect",
    resource=dl.TriggerResource.ITEM,
    actions=dl.TriggerAction.CREATED,
    filters=filters
)

Trigger the Service

Once the service is deployed, we can create a trigger to run it automatically when a certain event occurs.
In our example we trigger the face-detection service whenever an item is uploaded to the platform.
Consider using other triggers or different ways to run you service:

  • Add the services to a FaaS-node in a pipeline, before annotation tasks
  • Use DQL trigger to run only on specific datasets, or in specific tasks
  • Run the service when an item is updated
Copy
Copied
self.package.artifacts.download(artifact_name=artifact_filename,
                                local_path=full_weight_path)

Uploading Model Weights as Artifacts

Large data files such as ML model weights can be too big to include in a package. These and other large files can be uploaded as artifact.