PTZ control by Object Tracking

TheNukem

New Member
Hi all.

I've been a lurker on these forums for a while, but recently I've not been able to sit by and watch.


So this is a call out to someone who has skills far beyond mine....and I can tell you that having read a LOT of posts on these kind of forums, that this is a plugin that people are crying out for.

It'll save $1,000s of dollars on these crappy PTZs that claim to have AI tracking. OBS can do it all.

There are 2 awesome plugins that basically need linking, but I'm not the one to do it. I'm techie AF, but not a coder.

https://github.com/glikely/obs-ptz/
AND
https://github.com/locaal-ai/obs-detect
This is how I see it flowing:
  1. OBS captures videoObject Detection Plugin processes frames.
  2. Detection script calculates object position → sends PTZ adjustments.
  3. PTZ Plugin translates this into VISCA commandsCamera moves.

a few challenges:
  • Latency: PTZ movement may lag behind fast-moving objects.
  • Smoothing: I'll need to implement dead zones (only move if object is >X pixels off-center).
  • Calibration: Fine-tune pan/tilt speeds for smooth tracking.

Assuming that 'detect' already calcs the x, y positioning, then that data needs to feed it into PTZ control with an instruction set along the lines of 'if object is 'z' number of pixels off a specific point, then move camera until object is within tolerance.

Like I say, I'm no coder, but I am an award winning film maker as well as doing lots of multi-cam (PTZs) streaming of live events, performances and conferences. If I can be of any help to you in the development process, please let me know.
 

glikely

Member
I’m all for it! obs-ptz already exposes a basic api for control by another plugin, and it is used by @norihiro ‘s Face Tracker plugin . Other plugins could use it too.

The API is a little basic though. I’m thinking about creating a common API that allows other plugins (e.g., @DistroAV ) to expose PTZ controls which can be driven by obs-ptz or other plugins. Object tracking would fit right with that.
 

AronOskarss

New Member
Second this!
i need this for my local raceclub, i could setup cameras to auto follow cars and then return to preset 1 if none found for 10 seconds or something like that. That would give me more time to focus on commentating!
Please someone make this happen. i will raise money for you if needed!
Detect plugin worked amazing on cars but no PTZ control.
 

TheNukem

New Member
SO,

As much as I've loved this idea as it is so far, I've run into another issue.

I have 'proper' PTZ cameras. Panasonic AW HE130s....4 of them and guess what (and I've known this forever)....they run on their own protocol. Not Onvif / Visca / Pelco.

This hasn't been much of a problem for me as I use Companion (BitFocus) to control and preset / recall etc and a touch screen windows machine to do all the quick 'touch (click) to centre on the cursor' kinda things. And that's ALL good...

BUT this isn't going to work so easily with OBS PTZ Control, because OBS PTZ Control....doesn't control /my/ PTZs. :-(
 
Top