RESEARCH DIRECTION BRIEF / PROPOSED AGENDA
Reliable multimodal learning with limited labels
Hang Gu · 顾航
Lecturer and teaching section head at 江苏财经职业技术学院; MSc in Computer Science (Data Science), Universiti Malaya.
I want to understand when adapting frozen multimodal representations is reliable, and when scarce validation labels make apparent improvements unstable. News classification provides a concrete setting for connecting representation choices with model-selection decisions.
This brief proposes experiments for discussion. The designs below are planned work, not completed experiments or performance results.
01
Two testable questions
Q1 / PROPOSED EXPERIMENT
Does frozen-CLIP adaptation remain useful after split and artifact controls?
- Compare
- A linear classifier on frozen CLIP features, learned prompts, and token-level co-attention fusion. Keep the encoder frozen and compare each addition with the same feature baseline.
- Control
- Define train, validation and test manifests before tuning. Group duplicate or related samples before splitting; inspect source and event cues. Use identical label budgets, paired seeds and tuning budgets for every method.
- Evaluate / fail
- Predefine macro-F1 and a paired uncertainty interval for the difference from the baseline, with source/event slices as diagnostics. An improvement claim fails if its interval includes zero or its direction reverses on predefined artifact-controlled conditions. Report those failures.
Q2 / PROPOSED EXPERIMENT
How does validation sampling change model selection and early stopping?
- Compare
- Fixed-epoch selection and validation-based early stopping over the same candidate runs. Predefine patience and minimum-improvement settings, then repeat selection using paired stratified validation subsets at planned label budgets.
- Control
- Hold training data, candidate checkpoints, seeds and the sealed test set fixed. Cache validation-pool predictions so sampling changes only the selection input. Keep each comparison's validation and search budget equal; lock the protocol before inspecting test scores.
- Evaluate / fail
- Predefine checkpoint-selection agreement, stopping-epoch dispersion and paired macro-F1 differences across resamples. Freeze the agreement threshold before running the study. A stability claim fails below that threshold; any train/test overlap or missing class coverage invalidates the affected comparison.
02
Existing preparation
My first-author FCP study includes frozen-CLIP adaptation, baseline comparisons, modality ablations and near-duplicate audits. KICP provides research implementation and external-asset requirements. Q1 proposes a further controlled comparison, beyond these existing preparations.
The DASFAA 2027 submission design uses 5 training seeds, 20 shared validation draws and 100 additional draws to distinguish validation composition, score weighting and epoch selection. This design informs Q2; the recorded status is submitted, not accepted.
KANO-derived learning tools with explicit early-stopping state, sample-aligned predictions and synthetic component checks. These support Q2's evaluation implementation; they do not establish a trained benchmark. Upstream KANO attribution is retained.
Research descriptions and submission status above refer to the September 2026 record.
03
Proposed sequence
- 01
Define the protocol
Verify data access and grouping, reproduce a minimal baseline, and freeze splits, budgets, selection rules and failure criteria.
- 02
Run paired comparisons
Study controlled adaptation first; reuse its fixed candidate runs to isolate validation-sampling effects and inspect reversals.
- 03
Report the limits
Publish permitted code and prediction summaries with uncertainty, ablations and failure examples. Refine the next question from those findings.