| 5 | 0 | 13 |
| 下载次数 | 被引频次 | 阅读次数 |
在地外星球探测等复杂控制环境中，操作人员必须处理大量的多模态信息。传统的静态界面及平视显示器等解决方案在处理这些日益复杂的环境时表现出信息密度过大，且开发迭代成本高，缺乏灵活性等局限。为了克服这些局限，本文提出一种自适应界面构建方法，该方法能够根据操作人员的任务上下文和组件库，动态构建界面信息层级。该方法结合大语言模型的语义分析和增强现实设备的场景感知能力，优先呈现关键数据，过滤无关细节，从而优化决策过程。通过对模拟任务进行评估，该方法在任务表现和用户满意度方面能够对操作员进行有效辅助，验证了将实时自适应混合现实系统与上下文感知框架相结合的有效性。
Abstract：In complex control environments such as extraterrestrial planet exploration, operators must handle a large amount of multimodal information. Traditional static interfaces and solutions like heads-up displays show limitations when dealing with these increasingly complex environments, including excessive information density, high development and iteration costs, and lack of flexibility. To address these challenges, an adaptive interface construction method is proposed in this paper, which dynamically builds interface information hierarchies based on the operator's task context and component library. This method combines the semantic analysis of large language models and the scene awareness capabilities of mixed reality(MR) devices to prioritize key data and filter irrelevant details, thereby optimizing the decision-making process. Evaluation through simulated tasks indicates that this method can significantly enhance operator performance and user satisfaction, validating the effectiveness of combining real-time adaptive MR systems with context-aware frameworks.
[1]LINDLBAUER D, FEIT AM, HILLIGES O. Context-Aware UI optimization for mixed reality[C/OL]. UIST, 2019. https://dl.acm.org/doi/10.1145/3332165.3347945.
[2]GEBHARDT C, OULASVIRTA A, HILLIGES O. Learning personalised label display strategies for mixed reality[C/OL]. UIST. 2019. https://dl.acm.org/doi/abs/10.1145/3332165.3347945.
[3]CHENG Y, YAN Y, YI X, et al. SemanticAdapt:optimization-based adaptation of mixed reality layouts leveraging virtual-physical semantic connections[C/OL]. The 34th Annual ACM Symposium on User Interface Software and Technology, 2021. https://dl.acm.org/doi/10.1145/3472749.3474750. DOI:10.1145/3472749.3474750.
[4]BELO J, SAHAMI A, PFEIFFER J, et al. AUIT:a toolkit for multi-objective optimization in xr application design[C/OL]. UIST,2022. https://dl.acm.org/doi/abs/10.1145/3332165.3347945.
[5]JOHNS. A Multi-Objective UI layout adjustment method for MR[C/OL].UIST, 2024. https://dl.acm.org/doi/abs/10.1145/3613904.3617892.
[6]LI Z, GEBHARDT C, INGLIN Y, et al. SituationAdapt:contextual ui optimization in mixed reality with situation awareness via LLM reasoning[C/OL]. Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, 2024. https://dl.acm.org/doi/10.1145/3654777.3676470. DOI:10.1145/3654777.3676470.
[7]郑燕,王璟,葛列众.自适应用户界面研究综述[J].航天医学与医学工程, 2015, 28(2):145-150. DOI:10.16289/j.cnki.1002-0837.2015.02.013.
[8]袁庆曙,王若楠,潘志庚,等.空间增强现实中的人机交互技术综述[J].计算机辅助设计与图形学学报, 2021, 33(3):321-332.DOI:10.3724/SP.J.1089.2021.18445.
[9]FENG W, ZHU W, FU T jui, et al. Layoutgpt:Compositional visual planning and generation with large language models[C/OL].Advances in Neural Information Processing Systems, 2024. https://proceedings.neurips.cc/paper_files/paper/2023/hash/3a7f9e485845dac27423375c934cb4db-Abstract-Conference.html.
[10]WANG B, LI G, LI Y. Enabling conversational interaction with mobile UI using large language models[C/OL]. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. New York:Association for Computing Machinery, 2023:1-17. https://dl.acm.org/doi/10.1145/3544548.3580895. DOI:10.1145/3544548.3580895.
[11]SAUER Y, SIPATCHIN A, WAHL S, et al. Assessment of consumer VR-headsets’ objective and subjective field of view(FoV)and its feasibility for visual field testing[J]. VR, 2022, 26(3):1089-1101.DOI:10.1007/s10055-021-00619-x.
[12]BENNETT FV. Apollo experience report:Mission planning for lunar module descent and ascent[R].Washington:NASA, 1972.
[13]李茂登,黄翔宇,徐超,等.天问一号火星探测器EDL过程自主导航技术[J].宇航学报, 2022, 43(1):11-20. DOI:10.3873/j.issn.1000-1328.2022.01.002.
[14]CALDWELL B, COOPER M, REID LG. Web content accessibility guidelines(WCAG)2.0[EB/OL].(2008-12-11)[2025-02-13]. https://www.w3.org/TR/WCAG20/.
[15]DEB K, PRATAP A, AGARWAL S, et al. A fast and elitist multiobjective genetic algorithm:NSGA-Ⅱ[J]. IEEE Trans Evol Comput, 2002,6(2):182-197. DOI:10.1109/4235.996017.
[16]YAO S, ZHAO J, YU D, et al. ReAct:synergizing reasoning and acting in language models[C/OL]. ICLR, 2023. https://par.nsf.gov/biblio/10451467-react-synergizing-reasoning-acting-language-models.
[17]陈晓皎,束云峰,汪睿涵,等.大语言模型驱动的UI评估系统[J].图学学报, 2024, 45(6):1178-1186. DOI:10.11996/JG.j.2095-302X.2024061178.
[18]WEI J, WANG X, SCHUURMANS D, et al. Chain-of-thought prompting elicits reasoning in large language models[J]. Adv Neural Inf Process, 2022, 35:24824-24837. DOI:10.48550/arXiv.2201.11903.
[19]BENNETT KB, POSEY SM, SHATTUCK LG. Ecological interface design for military command and control[J]. J Cogn Eng Decis Mak,2008, 2(4):349-385. DOI:10.1518/155534308X377829.
[20]PAUZIÉA. A method to assess the driver mental workload:the driving activity load index(DALI)[J]. IET Intell Transp Syst, 2008, 2(4):315-322. DOI:10.1049/iet-its:20080023.
基本信息:
DOI：10.16289/j.cnki.1002-0837.2026.02007
中图分类号:V448
引用信息:
[1]陈晓皎,汪睿涵,戚温如,等.基于多模态上下文的空间自适应界面构建方法[J].航天医学与医学工程,2026,37(02):147-152.DOI:10.16289/j.cnki.1002-0837.2026.02007.
基金信息:
人因工程全国重点实验室基金（HFNKL2024W03）; 国家自然科学基金（52205290）
2025-06-19
2025
2025-10-15
2025-11-25
2025
1
2026-04-25
2026-04-25