Codeproject.AI

CodeProject.AI Server is a self-hosted, free and Open Source Artificial Intelligence Server for any platform, any language. Just like you would install a database server to provide data storage, you install CodeProject.AI Server to provide AI services.

It can be installed locally, requires no off-device or out of network data transfer, and is easy to use.

Head over to the CodeProject.AI Documentation for installation instructions.

Configuration

Configuration example

codeprojectai:
  host: codeprojectai
  port: 32168
  object_detector:
    cameras:
      viseron_CHANGEME_camera:
        fps: 1
        log_all_objects: true
        labels:
          - label: person
            confidence: 0.8
            trigger_recorder: true
          - label: cat
            confidence: 0.8
        zones:
          - name: zone1
            coordinates:
              - x: 0
                y: 500
              - x: 1920
                y: 500
              - x: 1920
                y: 1080
              - x: 0
                y: 1080
            labels:
              - label: person
                confidence: 0.8
                trigger_recorder: true
        mask:
          - coordinates:
              - x: 400
                y: 200
              - x: 1000
                y: 200
              - x: 1000
                y: 750
              - x: 400
                y: 750
      camera_two:
        fps: 1
        labels:
          - label: person
            confidence: 0.7
            trigger_recorder: true
          - label: cat
            confidence: 0.8
            trigger_recorder: false
  face_recognition:
    save_unknown_faces: true
    cameras:
      camera_two:
    labels:
      - person
  license_plate_recognition:
    cameras:
      camera_one:
    labels:
      - vehicle
      - car
      - truck
    known_plates:
      - ABC123

codeprojectaimap required

CodeProject.AI configuration.

hoststring required

IP or hostname to your CodeProject.AI server.

portinteger (optional, default: 32168)

Port to your CodeProject.AI server.

Lowest value: 1024

Highest value: 49151

timeoutinteger (optional, default: 10)

Timeout for requests to your CodeProject.AI server.

object_detectormap (optional)

Object detector domain config.

camerasmap required

Camera-specific configuration. All subordinate keys corresponds to the camera_identifier of a configured camera.

<CAMERA_IDENTIFIER>map required

Camera identifier. Valid characters are lowercase a-z, numbers and underscores.

fpsfloat (optional, default: 1)

The FPS at which the object detector runs.
Higher values will result in more scanning, which uses more resources.

Lowest value: 0

scan_on_motion_onlyboolean (optional, default: true)

When set to true and a motion_detector is configured, the object detector will only scan while motion is detected.

labelslist (optional)

A list of labels (objects) to track.

labelstring required

The label to track.

confidencefloat (optional, default: 0.8)

Lowest confidence allowed for detected objects. The lower the value, the more sensitive the detector will be, and the risk of false positives will increase.

Lowest value: 0

Highest value: 1

height_minfloat (optional, default: 0)

Minimum height allowed for detected objects, relative to stream height.

Lowest value: 0

Highest value: 1

height_maxfloat (optional, default: 1)

Maximum height allowed for detected objects, relative to stream height.

Lowest value: 0

Highest value: 1

width_minfloat (optional, default: 0)

Minimum width allowed for detected objects, relative to stream width.

Lowest value: 0

Highest value: 1

width_maxfloat (optional, default: 1)

Maximum width allowed for detected objects, relative to stream width.

Lowest value: 0

Highest value: 1

trigger_recorderboolean (optional, default: true)

If set to true, objects matching this filter will start the recorder.

require_motionboolean (optional, default: false)

If set to true, the recorder will stop as soon as motion is no longer detected, even if the object still is. This is useful to avoid never ending recordings of stationary objects, such as a car on a driveway

max_frame_agefloat (optional, default: 2)

Drop frames that are older than the given number. Specified in seconds.

Lowest value: 0

log_all_objectsboolean (optional, default: false)

When set to true and loglevel is DEBUG, all found objects will be logged, including the ones not tracked by labels.

masklist (optional)

A mask is used to exclude certain areas in the image from object detection.

coordinateslist required

List of X and Y coordinates to form a polygon

Minimum items: 3

xinteger required

X-coordinate (horizontal axis).

yinteger required

Y-coordinate (vertical axis).

zoneslist (optional)

Zones are used to define areas in the cameras field of view where you want to look for certain objects (labels).

namestring required

Name of the zone. Has to be unique per camera.

coordinateslist required

List of X and Y coordinates to form a polygon

Minimum items: 3

xinteger required

X-coordinate (horizontal axis).

yinteger required

Y-coordinate (vertical axis).

labelslist (optional)

A list of labels (objects) to track.

labelstring required

The label to track.

confidencefloat (optional, default: 0.8)

Lowest confidence allowed for detected objects. The lower the value, the more sensitive the detector will be, and the risk of false positives will increase.

Lowest value: 0

Highest value: 1

height_minfloat (optional, default: 0)

Minimum height allowed for detected objects, relative to stream height.

Lowest value: 0

Highest value: 1

height_maxfloat (optional, default: 1)

Maximum height allowed for detected objects, relative to stream height.

Lowest value: 0

Highest value: 1

width_minfloat (optional, default: 0)

Minimum width allowed for detected objects, relative to stream width.

Lowest value: 0

Highest value: 1

width_maxfloat (optional, default: 1)

Maximum width allowed for detected objects, relative to stream width.

Lowest value: 0

Highest value: 1

trigger_recorderboolean (optional, default: true)

If set to true, objects matching this filter will start the recorder.

require_motionboolean (optional, default: false)

image_sizeinteger (optional)

Frames will be resized to this width and height before inference to save computing power. Resizing is done by adding black borders to the image to keep the aspect ratio.

custom_modelstring (optional, default: ipcam-general)

Name of a custom CodeProject.AI model.

face_recognitionmap (optional)

Face recognition domain config.

camerasmap required

Camera-specific configuration. All subordinate keys corresponds to the camera_identifier of a configured camera.

<CAMERA_IDENTIFIER>map required

Camera identifier. Valid characters are lowercase a-z, numbers and underscores.

labelslist (optional)

A list of labels that when detected will be sent to the post processor. Applies only to this specific camera.

labelslist (optional)

A list of labels that when detected will be sent to the post processor. Applies to all cameras defined under cameras.

face_recognition_pathstring (optional, default: /config/face_recognition/faces)

Path to folder which contains subdirectories with images for each face to track.

save_unknown_facesboolean (optional, default: false)

If true, any unrecognized face will be saved to the folder specified in unknown_faces_path. You can then move this image to the folder of the correct person to improve accuracy.

unknown_faces_pathstring (optional, default: /config/face_recognition/faces/unknown)

Path to folder where unknown faces will be stored.

expire_afterfloat (optional, default: 5)

Time in seconds before a detected face is no longer considered detected.

Lowest value: 0

trainboolean (optional, default: true)

Train CodeProject.AI to recognize faces on Viseron start. Disable this when you have a good model trained.

min_confidencefloat (optional, default: 0.6)

Minimum confidence for a face to be considered a match.

Lowest value: 0

Highest value: 1

license_plate_recognitionmap (optional)

License plate recognition domain config.

camerasmap required

Camera-specific configuration. All subordinate keys corresponds to the camera_identifier of a configured camera.

<CAMERA_IDENTIFIER>map required

Camera identifier. Valid characters are lowercase a-z, numbers and underscores.

labelslist (optional)

A list of labels that when detected will be sent to the post processor. Applies only to this specific camera.

labelslist (optional)

A list of labels that when detected will be sent to the post processor. Applies to all cameras defined under cameras.

known_plateslist (optional)

List of known license plates. Each plate will have its own sensor.

min_confidencefloat (optional, default: 0.6)

Minimum confidence for a license plate detection.

Lowest value: 0

Highest value: 1

expire_afterfloat (optional, default: 5)

Time in seconds before a plate recognition expires.

Lowest value: 0

Face recognition

Face recognition runs as a post processor when a specific object is detected.

Labels

Labels are used to tell Viseron when to run a post processor.

Any label configured under the object_detector for your camera can be added to the post processors labels section.

note

Only objects that are tracked by an object_detector can be sent to a post_processor. The object also has to pass all of its filters (confidence, height, width etc).

Train

On startup images are read from face_recognition_path and a model is trained to recognize these faces.
The folder structure of the faces folder is very strict. Here is an example of the default one:

/config
|── face_recognition
|   └── faces
|       ├── person1
|       |   ├── image_of_person1_1.jpg
|       |   ├── image_of_person1_2.png
|       |   └── image_of_person1_3.jpg
|       └── person2
|       |   ├── image_of_person2_1.jpeg
|       |   └── image_of_person2_2.jpg

danger

You need to follow this folder structure, otherwise training will not be possible.

License plate recognition

License plate recognition runs as a post processor when a specific object is detected.

You can track known license plates by adding them to the known_plates list in the configuration. Known plates will be reported as binary sensors.

There is also a sensor entity that reports all the plates that have been detected. This can be used to trigger actions in other platforms, eg Home Assistant, when an unknown plate is detected.

Object detector

An object detector scans an image to identify multiple objects and their position.

tip

Object detectors can be taxing on the system, so it is wise to combine it with a motion detector

Labels

Labels are used to tell Viseron what objects to look for and keep recordings of. The available labels depends on what detection model you are using.

The max/min width/height is used to filter out any unreasonably large/small objects to reduce false positives.
Objects can also be filtered out with the use of an optional mask.

Zones

Zones are used to define areas in the cameras field of view where you want to look for certain objects (labels).
Say you have a camera facing the sidewalk and have labels setup to record the label person.
This would cause Viseron to start recording people who are walking past the camera on the sidewalk. Not ideal.
To remedy this you define a zone which covers only the area that you are actually interested in, excluding the sidewalk.

codeprojectai:
  object_detector:
    cameras:
      camera_one:
        ...
        zones:
          - name: sidewalk
            coordinates:
              - x: 522
                y: 11
              - x: 729
                y: 275
              - x: 333
                y: 603
              - x: 171
                y: 97
            labels:
              - label: person
                confidence: 0.8
                trigger_recorder: true

tip

See Mask for how to get the coordinates for a zone.

Mask

Masks are used to exclude certain areas in the image from object detection. If a detected object has its lower portion inside of the mask it will be discarded.

The coordinates form a polygon around the masked area.
To easily generate coordinates you can use a tool like image-map.net.
Just upload an image from your camera, choose the Poly shape and start drawing your mask.
Then click Show me the code! and adapt it to the config format.
Coordinates coords="522,11,729,275,333,603,171,97" should be turned into this:

codeprojectai:
  object_detector:
    cameras:
      camera_one:
        ...
        mask:
          - coordinates:
              - x: 522
                y: 11
              - x: 729
                y: 275
              - x: 333
                y: 603
              - x: 171
                y: 97

Paste your coordinates here and press Get config to generate a config example

Codeproject.AI

Configuration​

Face recognition​

Labels​

Train​

License plate recognition​

Object detector​

Labels​

Zones​

Mask​

Configuration

Face recognition

Labels

Train

License plate recognition

Object detector

Labels

Zones

Mask