This project is an edge vision system delivered to a hospital rehabilitation department. The goal was not simply to recognize a camera frame, but to turn a clinician's experience-based gait and balance assessment into a local workflow that can be captured, computed, explained, and reviewed.
Workflow
The clinician selects a patient and a test in the assessment UI, then a USB camera starts recording. Jetson Nano performs the entire inference pipeline inside the hospital, so patient video is not uploaded. OpenCV manages frame capture and timestamps; the model emits 17 human keypoints with confidence values; a rules engine converts the sequence into gait, balance, and sit-to-stand metrics. The system produces a Tinetti-assisted score and report with trajectory evidence, low-confidence notices, and per-item results.
Implementation
Capture and preprocessing
Frames enter a bounded queue so a slow inference step cannot grow memory without limit. CLAHE improves local contrast, while the L channel in LAB color space is normalized to reduce fluorescent and window-light shifts. Every frame keeps its timestamp and keypoint confidence; cadence and trajectory calculations cannot rely on frame index alone.
Pose inference
YOLOv11-Pose was selected because its single-stage detector performs localization and keypoint regression together, which fits Jetson Nano's low-power envelope. The output contains 17 standard body nodes with x, y, and confidence. Low-confidence points are not silently filled in to make a complete-looking skeleton; they remain marked as unreliable and surface in the UI and report.
TensorRT optimization
The first PyTorch implementation hit the Jetson Nano 4GB shared-memory wall. The final path exports the PyTorch model to ONNX, builds an FP16 TensorRT engine on the Nano itself, and loads that fixed engine from the local inference service. The engine must be built on the target device because the PC TensorRT version is incompatible with the Nano's 8.x runtime.
Native inference was about 14 FPS; TensorRT raised it to roughly 26–29 FPS. Memory use fell from about 2.7GB to about 1.2GB, while keypoint coordinate error stayed below 6 pixels. That left enough headroom for capture, rule evaluation, and report rendering.
Translating a scale into rules
The hard part of Tinetti was not arithmetic but semantics. “Gait symmetry” is not equality at one frame; it is the deviation between left and right stride lengths over consecutive gait cycles. “Trunk stability” is represented by the standard deviation of the shoulder midpoint's lateral trajectory. “A successful stand” is a transition from a seated keypoint configuration to a standing one, not a single knee angle.
Continuity required a rewrite. The first version treated near-zero ankle velocity as a pause. Clinicians pointed out that an older adult may slow briefly to adjust balance without losing rhythm. The final implementation uses step-frequency standard deviation: stable cadence has low variance, while hesitation and broken rhythm increase it. Each threshold is reviewed against real cases instead of being chosen from developer intuition.
Field problems
Lighting. Wrists and ankles are small targets, and warm afternoon light reduced their confidence. CLAHE, LAB normalization, and an explicit low-confidence notice prevent a complete-looking but unreliable result.
Memory. CPU and GPU share 4GB, with roughly 1.5GB already used by the system. An FP16 engine, fixed input size, and bounded frame queue keep the device stable.
Clinical semantics. “Walks steadily” must become a measurable combination of trunk sway, cadence variation, and step symmetry. The project used shadowing, replayed examples, and threshold review to turn clinical language into explainable metrics.
Body-shape shift. Kyphosis, joint deformities, and large height differences make generic pose data a poor match. About 200 in-hospital clips were used for small-scale domain adaptation, with hip-knee-ankle bone-length constraints to trigger correction or reduce confidence when proportions are implausible.
Delivery
The system completed a two-week hospital trial and passed rehabilitation-department acceptance. Agreement with clinician scoring was about 89% for key actions, a single assessment averaged about six minutes, and the system ran continuously for 72 hours without a crash. It did not replace clinicians. It turned repetitive observation, recording, and calculation into a traceable assistant: the model extracts evidence, rules calculate metrics, and clinicians make the final interpretation.