Navio: Техническое собеседование

Во время обучения dropout случайно обнуляет часть активаций, а при выводе модели отключается. Масштабирование сохраняет одинаковое математическое ожидание величины активаций в обоих режимах.

Dropout — способ регуляризации, применяемый во время обучения. На каждом прямом проходе он независимо обнуляет долю p активаций. Благодаря этому модель не может постоянно опираться на один и тот же набор нейронов и ведёт себя как ансамбль множества подсетей.

При выводе модели dropout отключают: предсказания должны быть детерминированными и использовать всю сеть. Без масштабирования ожидаемая величина активаций различалась бы при обучении и при выводе. В распространённом варианте inverted dropout сохранённые активации во время обучения делят на 1 - p. Поэтому при выводе дополнительное умножение уже не требуется.

В другой конвенции активации при обучении не масштабируют, а при выводе умножают на 1 - p. На собеседовании важнее объяснить совпадение математических ожиданий, чем запомнить одну конкретную конвенцию.

Yes. Accuracy only checks the thresholded class, while BCE also penalizes confidence. A few confidently wrong examples can increase loss while more examples cross the threshold correctly.

This can happen because accuracy and BCE measure different things. Accuracy is threshold-based: after choosing a threshold such as 0.5, it only counts whether each prediction is on the correct side. BCE uses the predicted probability, so confidence matters.

Suppose more validation examples move from 0.49 to 0.51 for the correct class. Accuracy improves. At the same time, a small number of mislabeled, shifted or hard examples can receive very confident wrong probabilities, such as 0.999 for the wrong class. BCE on those examples can grow sharply and dominate the average loss.

Plausible causes include label noise, validation/train domain shift, overconfident miscalibration, or a distribution slice where the model becomes confidently wrong. A strong answer proposes inspecting the confusion matrix, per-slice loss, calibration curves and mislabeled examples.

BatchNorm is the classic issue: each process sees only its local mini-batch, so statistics can diverge. Use SyncBatchNorm, larger effective batches, different normalization, or accept the approximation.

BatchNorm depends on batch statistics. In DDP, each process usually receives only a shard of the global batch, so ordinary BatchNorm computes mean and variance from the local shard. If the per-GPU batch is small or non-representative, the normalization can be noisy or inconsistent.

One common fix is synchronized batch normalization, such as PyTorch SyncBatchNorm, which synchronizes statistics across processes. Another path is to use normalization layers that do not depend on cross-sample batch statistics, such as LayerNorm or GroupNorm, depending on the architecture. Some teams also simply tolerate local BatchNorm when the per-device batch is large enough and metrics are stable.

The production answer should include the tradeoff: synchronization costs communication and can slow training, so it is not automatically the best choice.

Optimizers compile a static graph for target hardware, fuse operators, choose kernels and reduce precision. INT8 needs calibration to map real activation ranges into 8-bit values without destroying accuracy.

ONNX is commonly used as an interchange format that describes a static computation graph and a set of operations. TensorRT can take such a graph and build an engine tuned for a specific NVIDIA GPU. The speedups come from operator fusion, kernel selection, memory-layout planning, constant folding, removing unnecessary work and using lower precision where safe.

FP16 quantization is often relatively straightforward because it keeps floating-point semantics with fewer bits. INT8 is more delicate: activations and weights must be mapped into a small integer range. Calibration uses representative data to estimate activation ranges or distributions and choose scales and zero points. Without this, outliers and poorly chosen ranges can destroy model quality.

A good production answer also mentions dynamic shapes and unsupported operators as common reasons conversions fail or become slower than expected.

It is technically possible for fully convolutional detectors, but quality can change because feature scales, anchors and small-object visibility shift. Fully connected heads may require fixed input size.

Whether inference at a different resolution is technically possible depends on the architecture. A fully convolutional detector can usually accept different spatial dimensions, subject to stride divisibility and implementation constraints. If the model has fixed-size fully connected layers, those layers break unless they are replaced or adapted.

Even when the forward pass works, model quality can degrade. The detector learned feature scales, anchors, receptive-field behavior and preprocessing assumptions at the training resolution. Downscaling can hurt small objects; upscaling can change calibration and increase compute. In production, teams usually train and validate on the same resolution used for deployment or run explicit multi-scale training/evaluation.

For an automotive perception role, the stronger answer connects resolution choices to latency, hardware budget and safety-critical recall.