Port the models, prove the system

1 · Map every Python contract to an executable node

Python remains excellent for training, rapid exploration and official devkit evaluation. A Python bridge can validate bring-up. The composed C++/TensorRT graph buys bounded queueing, observable per-link failures, shared GPU context and concurrency; it must earn complexity through measurement rather than assumption.

2 · Define a numerical contract before enabling TensorRT

The reference point-cloud range is $[-54,-54,-5,54,54,3]$ m; voxel size $[.075,.075,.2]$ m yields a $1440\times1440\times40$ grid, downsampled by 8 to $180\times180$ BEV. LSS depth samples $1$ to $60$ m by $.5$ m. Assert $grid/8=bev_{H,W}$ at startup. Misinterpreting $(z,y,x)$ voxel order, changing the point time sign, swapping length/width, or treating a box bottom as center produces plausible pictures and wrong trajectories.

1. Graph-only: synthetic input, stub backend, one container, 11 model/health nodes, no unexpected frame drops.
2. Preprocess parity: same nuScenes frame, Python vs ROS FP16 image and $[N,5]$ point tensors; compare with a defined absolute tolerance.
3. Encoder parity: same input into both, compare camera and LiDAR BEV features by numerical errors and per-channel cosine similarity; identify depth/voxel indexing errors.
4. Detection parity: freeze confidence/NMS and compare box count, center, yaw, size, class/score.
5. Tracking parity: inject identical detections into both trackers for a complete scene, compare association and lifecycle/ID sequence.
6. Only then performance: FP16/TensorRT, pinned uploads, preallocated buffers, stream overlap, CUDA graphs. Re-run quality after every optimization.

3 · Segmentation is three different engineering problems

The six BEV map channels are drivable area, pedestrian crossing, walkway, stop line, carpark and divider. They overlap: use per-channel sigmoids, not mutually exclusive softmax (a crosswalk is also drivable). BevGrid is channel-first: $idx=cHW+rW+col$, with forward $x$ mapping to column and left $y$ to row. Publish a multi-channel probability grid and a separate planner-compatible single-channel drivable OccupancyGrid. Near a free/occupied decision boundary represent unknown rather than assert free. Document whether occupancy values encode free=0 / occupied=100 / unknown=-1 and when a grid is stale.

If image segmentation is needed, reuse GPU images; publish compact per-view labels at a lower rate on a low-priority stream, and undistort → infer depth → transform → rasterize before calling it a BEV map. A 200×200×16 voxel grid with 18 FP16 class channels is about 23 MB per frame; keep that on the GPU and downsample only for debugging. Example planning budget: detection head ~15 ms high priority, BEV segmentation ~8 ms lower priority, optional occupancy ~25 ms and six-camera 2D head ~20 ms; enabling all may exceed the 100 ms period. Profile on hardware rather than assuming stream overlap makes compute free.

Evaluate BEV with per-class IoU $IoU_c=TP_c/(TP_c+FP_c+FN_c)$ (also range slices), image boundaries with boundary F1, occupancy using occupied-voxel/ray IoU. A mean dominated by drivable road surface says little about rare stop-line misses or ramps.

4 · Debug from the first violated contract

ros2 node list
ros2 topic info /lidar/points --verbose
ros2 topic echo /lidar/points --qos-reliability best_effort --once
ros2 component list
ros2 param dump /det_head
ros2 run perception_py latency_report
ros2 topic echo /perception/status

Trace callback dispatch and publish-to-callback delay with ros2_tracing/LTTng (or CARET to reconstruct chains); correlate stamps to CUDA/NVTX ranges using Nsight Systems. Look for unexpected memcpy between BEV stages, no camera/LiDAR kernel overlap, steady-state cudaMalloc, device-wide synchronization and unpinned uploads. Profiling within a callback alone misses executor waiting: include capture and end-to-end age.

5 · A test pyramid that can falsify the design

colcon build --symlink-install --cmake-args -DPERCEPTION_WITH_CUDA=OFF
colcon test --event-handlers console_direct+
colcon test-result --verbose
ros2 launch perception_bringup full_system.launch.py backend:=stub source:=synthetic

A successful CPU-stub latency test proves plumbing, not the real BEVFusion latency or accuracy. Run thermal and sustained-load tests on the deployment SoC; require quality and latency together. Route rosbag recording to a separate low-priority process; record sensor stamps, raw packets, calibrated state, outputs, status, per-stage latency and evaluation results if GT exists. Do not record process-local CUDA pointer topics: they cannot be meaningfully replayed. Include negative tests for QoS mismatches and lifecycle restart, not just sunny-day scenes.

6 · Explain it at senior level: challenge drills

For every answer give a choice, the alternative you rejected, a failure mode and a measurement. Practice these aloud:

1. Six cameras + LiDAR: justify QoS for each link; why is a reliable 6 MB sweep often worse than one lost sweep?
2. Why does a multithreaded executor sometimes serialize an entire node? Show explicit callback groups, affinity and a service deadlock fix.
3. Distinguish intra-process move, shared fan-out, loaned SHM and a GPU handle; prove each rather than labeling everything “zero copy.”
4. Whiteboard the Python SensorFrame-to-BEVFusion-to-tracker mapping; where do temporal calibration and per-point LiDAR offsets come from?
5. Why is GNC not in the GPU container? Which topics and freshness limits form its contract, and how does it stop if perception dies?
6. Why does BEV map segmentation use sigmoids? When is 3D occupancy worth its 23 MB per frame?
7. mAP improved, but pedestrian recall at 40 m got worse. Which number limits speed? How do you find a threshold vs sensor vs calibration failure?
8. MOTA .71, recall .88, IDF1 field .90: which definitions are actually implemented? Does this dashboard compute sequence-level IDF1?
9. A LiDAR-only fallback “works” on a sunny dataset; how do you test spray, low-return pedestrians and sensor common-mode loss?

7 · Source guide: the complete ROS 2 study set

The tutorial's pages map to all twelve source chapters. Continue at the section that answers your next question: