Exposed: Video Systems Trick Unsuspecting Researchers!
or "What every vision researcher should know about video equipment"
Table of Contents
- Oh, come on, of course video is 30 images per second!
- Gee, that must mean the sampling interval is 1/60 second.
- Pausing the VCR still gives me a full-size image -- that must mean that it is full resolution!
- Since the VCR just records and plays video, the digitizer can't tell the difference.
- Well my VCR uses time code and even tells me the current frame -- that can't be wrong!
- 30 (29.97) frames per second
- 2 fields per frame, interlaced
- Test: image a simple LED flashing at 30 Hz -- ON for one
field, OFF for the next:
- Doubles one field in Freeze Frame (Pause) or Slow Play
- not programmable from computer
- more obvious: Resolution loss (VHS, S-VHS, Umatic, etc.)
- less obvious: sync quality!
- Displays (monitors, televisions, etc.) robust
- Digitizers less robust
- Result: looks fine on monitor, but digitizes "poorly"
(dropped lines, offset lines, "waviness", etc.)
- extremely inconspicuous:...[more later]
- Vertical Interval Time Code (VITC): embedded in blanking portion of video signal (vs. Longitudinal Time Code, LTC, stored on a separate audio channel)
- VCRs that read VITC: trustworthy? Press "Stop" and see:
- Observations:
- VITC looks like a video line
- VCR (in PLAY mode) generates "reasonable" video
- A digitizer simply captures video lines
- Conclusion:
- Make the digitizer capture VITC as video
- Perform our own interpretation (ouch?)
- Trial 1: OK
- Trial 2: Field 1 data in Field 2? Field 2 data in Field 1?
- Trial 3: OK
- ...
- Summary: about half are OK, half are "field swapped"
- Huh?
- Suspicion: recording VCRs ignore incoming sync,
storing first stable field as Field 1
It WORKS!
- Camera 1: OK
- Camera 2: OK
- ...
- Camera 24: OK
- Camera 25: OK - no, problem - uh, no, OK -- huh?
- ...
- Camera 51: definitely not OK
- LTC-to-VITC translator: "re-shapes" LTC
after ~25 loops, begins to degrade
Solution: "Fan out" LTC to multiple translators rather than daisy-chaining them all into one serial line
- Synchronous: 51 cameras taking pictures simultaneously
- Fast: Full video capture
- High Capacity: hours (not minutes or seconds) of recording
- Extensible: camera + VITC inserter + VCR
- Inexpensive (per channel): $500 + $250 + $350 = $1100
- Factors:
- Frame Integration vs. Field Integration
- Synchronization of Cameras
- Re-Synchronization of Images after Digitization
- Need: External device, changing state with sync
- Solution: Build a frame/field counter (ugh!)
- Flashing LEDs reveal mode and field
- Field Mode: frame-rate LED imaged ON, then OFF
- Frame Mode: half frame-rate LEDs imaged same
(ON-ON), then opposite (ON-OFF or OFF-ON)
Actual Images
Validation: Synchronization
- Camera Sync: Any freely running cameras?
- Flashing LEDs ON always
- Multiple numbers visible in each field
- Re-sync after Digitization: Cam (i, t0) != Cam(j, t0)?
- Display shows different numbers in different cameras
- Offset between real VITC and display changes from
one camera to another (expected to be constant)
- Virtualized Reality (original motivation and in use)
- Dynamic Scene Structure Recovery (in use)
- Motion Analysis [single view or multi-view] (in use)
- Human motion tracking and recognition
- Analysis and recognition of human facial expressions
- Assembly Plan from Observation (in use)
- View Interpolation (planned)
- Photometric Modelling?
