H. Zhang, Q. Zhang, Q. Cao, H. Fan, Y. Huang, K. Sha, and Y. Feng.
TriAttention: Unified Sparse Mixed-Precision Attention for Long-Context LLM Inference.
arXiv preprint, 2026.
@misc{zhang2026triattentionunifiedsparsemixedprecision,
title={TriAttention: Unified Sparse Mixed-Precision Attention for Long-Context LLM Inference},
author={Hanzhi Zhang and Qiao Zhang and Qinglei Cao and Heng Fan and Yan Huang and Kewei Sha and Yunhe Feng},
year={2026},
eprint={2611.07256},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2611.07256},
}
Sparsity and quantization reduce the cost of attention by skipping interactions and lowering the precision of those retained. However, useful interactions differ in their precision requirements: some tolerate low-precision scores while remaining sensitive to removal. We therefore introduce TriAttention, which assigns query-key (QK) tiles to skipping, 4-bit, or 8-bit computation, using 4-bit to retain quantization-tolerant interactions at lower cost. To represent compactly, we encode each head's computation regions with two diagonal-distance boundaries, keeping policy storage independent of context length. Offline calibration then evaluates candidate boundary pairs using normalized projected-output errors and complete-tile execution costs. Using these measurements, a model-wide allocator selects each head's boundaries to minimize calibration error under a shared computation budget. During prefill, the selected boundaries generate precision-grouped traversals for a fused kernel that skips omitted tiles and combines contributions from both QK precisions into one normalized attention output. The quantized keys and values prepared during prefill are then reused in a compressed key-value (KV) cache for decoding, reducing repeated reads of prompt data. Experiments show that TriAttention achieves end-to-end generation speedups of up to 7.88x over the Torch attention and 2.65x over the BF16 baseline. Meanwhile, TriAttention maintains quality comparable to the original models on general-purpose and long-context tasks. These results show how a compact retention-and-precision policy translates local sensitivities into attention computation and repeated KV-cache access. The implementation is https://github.com/HanzhiZhang-Ulrica/TriAttention.
H. Zhang, Q. Zhang, Q. Cao, H. Fan, Y. Huang, K. Sha, and Y. Feng.
TileQuant: Tile as the Precision-Allocation Unit for LLM Mixed-Precision Post-Training Quantization.
arXiv preprint, 2026.
@misc{zhang2026tilequanttileprecisionallocationunit,
title={TileQuant: Tile as the Precision-Allocation Unit for LLM Mixed-Precision Post-Training Quantization},
author={Hanzhi Zhang and Qiao Zhang and Qinglei Cao and Heng Fan and Yan Huang and Kewei Sha and Yunhe Feng},
year={2026},
eprint={2611.07217},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2611.07217},
}
Post-training quantization (PTQ) reduces large language model (LLM) deployment costs, but preserving quality requires accounting for uneven weight sensitivity. Uniform PTQ uses one weight bit-width, importance-aware PTQ adjusts scales or reconstructs weights within that width, and column-wise mixed precision retains selected columns at higher precision. However, sensitivity varies within columns, so whole-column retention also protects less sensitive weights, while runtime outlier detection adds decoding overhead. We therefore introduce TileQuant, a training-free framework that assigns precision to hardware-aligned 2D weight tiles. Offline calibration provides input-activation-aware scoring from forward statistics for lower calibration cost and gradient-informed scoring from loss sensitivity for stronger quality preservation. Within each projection's retention budget, the highest-scoring tiles remain at 16-bit and the rest use 8-bit. The resulting map guides reconstruction into compact weight buffers, which the kernel reuses for native integer and 16-bit computation with local accumulation to reduce memory traffic. Experiments show that TileQuant reaches 2.66x the original model's generation throughput. Across language modeling, reasoning, long-context retrieval, and question answering, it preserves quality and surpasses quantized and original models on multiple tasks. These results support selecting precision units that capture local sensitivity while enabling efficient hardware execution. The implementation is: https://github.com/HanzhiZhang-Ulrica/TileQuant
H. Zhang, Q. Zhang, Q. Cao, H. Fan, Y. Huang, K. Sha, and Y. Feng.
AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation.
arXiv preprint, 2026.
@misc{zhang2026alignquanttilealignedmixedprecisionquantization,
title={AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation},
author={Hanzhi Zhang and Qiao Zhang and Qinglei Cao and Heng Fan and Yan Huang and Kewei Sha and Yunhe Feng},
year={2026},
eprint={2610.07457},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2610.07457},
}
Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU storage and computation units. This precision-boundary mismatch limits the translation of compression into practical acceleration. We introduce AlignQuant, a post-training quantization method that uses GPU-compatible two-dimensional weight tiles as the common unit of precision allocation, compact storage, and execution. This shared partition lets precision follow sensitivity within output channels. Joint prefill/decode calibration scores precision reductions using projection-output perturbations weighted by language-model loss gradients under quantized activations. Phase-normalized scores prioritize higher precision for tiles important to either phase under a model-wide weight-storage budget. Each tile stores one selected representation, while phase-specialized kernels reuse the packed model and expand lower-bit weights for INT8 computation with 8-bit activations. Across four LLMs spanning 3B to 14B parameters, AlignQuant achieves up to 2.50× generation speedup over BF16 while preserving model quality. Evaluations further cover three GPUs and contexts up to 64K tokens. These results show that local precision flexibility and regular GPU execution can coexist through a shared tile unit. The implementation is available at https://github.com/HanzhiZhang-Ulrica/AlignQuant.
H. Zhang, Q. Zhang, Q. Cao, H. Fan, Y. Huang, K. Sha, and Y. Feng.
TileMix: Tile-Centric Mixed-Precision Attention for LLM Inference Acceleration.
arXiv preprint, 2026.
@misc{zhang2026tilemixtilecentricmixedprecisionattention,
title={TileMix: Tile-Centric Mixed-Precision Attention for LLM Inference Acceleration},
author={Hanzhi Zhang and Qiao Zhang and Qinglei Cao and Heng Fan and Yan Huang and Kewei Sha and Yunhe Feng},
year={2026},
eprint={2608.17336},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2608.17336},
}
Long-context prefill in large language models (LLMs) incurs substantial computation and memory traffic because dense self-attention computes quadratic query-key scores. Existing methods either use a uniform low-precision path or select token interactions, leaving spatial precision routing over hardware-aligned score tiles outside fused dense attention. We introduce TileMix, a tile-centric precision-routing kernel that makes numerical precision an executable spatial decision over score-tile groups within fused dense attention. TileMix partitions the attention matrix into hardware-aligned score tiles, packs routing decisions into compact bitmasks, and dispatches each tile group through FP16 or INT8 score computation while both paths update a shared online-softmax state. Scalable precision grouping lets each routing bit govern multiple adjacent key tiles, preserving hardware-aligned compute tiles and compact metadata at long contexts. By routing all legal tile groups, TileMix preserves dense token connectivity, requires no training, and supports grouped-query attention, variable-length batches, and INT8 key/value caches. Across LongEval, LV-Eval, and A100 prefill benchmarks on LLaMA, Qwen, and Vicuna, TileMix recovers long-context quality lost under uniform INT8 and improves prefill throughput over FP16, yielding a controllable accuracy-efficiency frontier across model families. The implementation is available at https://github.com/HanzhiZhang-Ulrica/TileMix.
B. Fan*, M. Li*, H. Zhang*, S. Dong, N. P. Mareedu, X. Liu, W. Shi, Y. Feng, Y. Huang, and H. Fan.
Towards Visual Query Segmentation in the Wild.
Advances in Neural Information Processing Systems (NeurIPS), 2026.
@misc{fan2026visualquerysegmentationwild,
title={Towards Visual Query Segmentation in the Wild},
author={Bing Fan and Minghao Li and Hanzhi Zhang and Shaohua Dong and Naga Prudhvi Mareedu and Xiaoqiong Liu and Weishi Shi and Yunhe Feng and Yan Huang and Heng Fan},
year={2026},
eprint={2603.08898},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2603.08898},
}
In this paper, we introduce visual query segmentation (VQS), a new paradigm of visual query localization (VQL) that aims to segment all pixel-level occurrences of an object of interest in an untrimmed video, given a visual query. Compared with existing VQL locating only the last appearance of a target using bounding boxes, VQS enables more comprehensive (i.e., all object occurrences) and precise (i.e., pixel-level masks) localization, making it more practical for real-world scenarios. To foster research on this task, we introduce VQS-4K, a large-scale benchmark dedicated to VQS. Specifically, VQS-4K consists of 4,111 videos with more than 1.3 million frames, and covers a diverse set of 222 object categories. Each video is paired with a visual query defined by a frame outside search video and its target mask, and annotated with spatial-temporal masklets corresponding to the queried target. To ensure high quality, all videos in VQS-4K are manually labeled with meticulous inspection and iterative refinement. To our best knowledge, VQS-4K is the first benchmark specifically designed for VQS. Furthermore, to stimulate future research, we present a simple yet effective method, named VQ-SAM, which extends SAM 2 by leveraging target-specific and background distractor cues from the video to progressively evolve the memory through a novel multi-stage framework with an adaptive memory generation (AMG) module for VQS, significantly improving the performance. In our experiments on VQS-4K, VQ-SAM achieves promising results and surpasses existing methods, evidencing its effectiveness. With VQS-4K and VQ-SAM, we expect to inspire more future research and applications on VQS. Our benchmark and code are released at https://github.com/VAIL-Fan/VQS.
H. Zhang, S. Anjum, H. Fan, W. Zheng, Y. Huang, and Y. Feng.
Poly-FEVER: A Multilingual Hallucination Detection Benchmark.
Findings of the Association for Computational Linguistics (EMNLP Findings), 2026.
@inproceedings{zhang-etal-2026-poly,
title = "Poly-{FEVER}: A Multilingual Hallucination Detection Benchmark",
author = "Zhang, Hanzhi and
Anjum, Sumera and
Fan, Heng and
Zheng, Weijian and
Huang, Yan and
Feng, Yunhe",
editor = "Lee, Hung-Yi and
Dev, Sunipa and
Li, Jessy and
Elliott, Desmond",
booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EMNLP} 2026",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.findings-emnlp.540/",
doi = "10.18653/2026.findings-emnlp.540",
pages = "11353--11369",
ISBN = "979-8-89176-462-0",
abstract = "We present Poly-FEVER, an open-source large-scale multilingual benchmark for hallucination detection through controlled factuality judgment in large language models. Poly-FEVER extends FEVER, Climate-FEVER, and SciFact to 77,973 Supported/Refuted claims aligned across 11 languages. We augment each claim with topic metadata from a 22-topic LDA model, enabling topic-aware analysis of multilingual factuality behavior. Translations are produced with Google Cloud Translation and validated by a targeted human audit, GEMBA, and COMETKiwi. We benchmark ChatGPT-3.5, LLaMA-2/3, Phi-3.5, and Mistral across three prompt families and evaluate robustness to model-generated rephrasing. We further analyze resource imbalance using web presence as a proxy for online claim availability, and evaluate retrieval-augmented factuality judgment with DPR over Wikipedia. Results show pronounced cross-lingual disparities under aligned claims, with stronger accuracy in high-resource languages such as English and Chinese and weaker accuracy in lower-resource languages such as Amharic and Tamil. Topic-aware prompting and retrieval provide selective gains, especially in Arabic and Amharic, while their effects vary across languages and can conflict with model priors. Poly-FEVER establishes a rigorous resource for responsible, language-adaptive hallucination detection and language-sensitive factuality evaluation. The proposed Poly-FEVER is available at: https://huggingface.co/datasets/HanzhiZhang/Poly-FEVER."
}
We present Poly-FEVER, an open-source large-scale multilingual benchmark for hallucination detection through controlled factuality judgment in large language models. Poly-FEVER extends FEVER, Climate-FEVER, and SciFact to 77,973 Supported/Refuted claims aligned across 11 languages. We augment each claim with topic metadata from a 22-topic LDA model, enabling topic-aware analysis of multilingual factuality behavior. Translations are produced with Google Cloud Translation and validated by a targeted human audit, GEMBA, and COMETKiwi. We benchmark ChatGPT-3.5, LLaMA-2/3, Phi-3.5, and Mistral across three prompt families and evaluate robustness to model-generated rephrasing. We further analyze resource imbalance using web presence as a proxy for online claim availability, and evaluate retrieval-augmented factuality judgment with DPR over Wikipedia. Results show pronounced cross-lingual disparities under aligned claims, with stronger accuracy in high-resource languages such as English and Chinese and weaker accuracy in lower-resource languages such as Amharic and Tamil. Topic-aware prompting and retrieval provide selective gains, especially in Arabic and Amharic, while their effects vary across languages and can conflict with model priors. Poly-FEVER establishes a rigorous resource for responsible, language-adaptive hallucination detection and language-sensitive factuality evaluation. The proposed Poly-FEVER is available at: https://huggingface.co/datasets/HanzhiZhang/Poly-FEVER.
M. Li, H. Zhang, H. Fan, J. Ding, and Y. Feng.
Harmful Factuality: LLMs Correcting What They Shouldn't.
Findings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL Findings), 2026.
@inproceedings{li-etal-2026-harmful,
title = "Harmful Factuality: {LLM}s Correcting What They Shouldn{'}t",
author = "Li, Mingchen and
Zhang, Hanzhi and
Fan, Heng and
Ding, Junhua and
Feng, Yunhe",
editor = "Demberg, Vera and
Inui, Kentaro and
Marquez, Llu{\'i}s",
booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EACL} 2026",
month = mar,
year = "2026",
address = "Rabat, Morocco",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.findings-eacl.46/",
doi = "10.18653/v1/2026.findings-eacl.46",
pages = "896--912",
ISBN = "979-8-89176-386-9",
abstract = "While Large Language Models (LLMs) are trained for factual accuracy, this objective can directly conflict with the critical demand for source fidelity. This paper isolates and formalizes this conflict as Harmful Factuality Hallucination (HFH): a previously overlooked failure mode where an LLM{'}s attempt to ``correct'' perceived source errors results in an output that is factually true but unfaithful to the input. Unlike traditional hallucination research focused on models generating falsehoods, we investigate the harm of misplaced correctness. We introduce a reproducible framework to elicit and measure HFH using controlled entity-level perturbations (both soft, embedding-based and hard, instruction-based) paired with strategic entity selection. Across summarization, rephrasing, and QA tasks, our evaluation of diverse LLMs reveals that HFH is a prevalent behavior that worsens with model scale. We identify three underlying mechanisms and demonstrate that a simple instructional prompt can reduce HFH rates by approximately 50{\%}. Our framework turns the abstract factuality{--}faithfulness tension into a measurable, actionable target for building more reliable LLM systems. Our code is publicly available at \url{https://github.com/ResponsibleAILab/Harmful-Factuality-Hallucination}."
}
While Large Language Models (LLMs) are trained for factual accuracy, this objective can directly conflict with the critical demand for source fidelity. This paper isolates and formalizes this conflict as Harmful Factuality Hallucination (HFH): a previously overlooked failure mode where an LLM's attempt to “correct” perceived source errors results in an output that is factually true but unfaithful to the input. Unlike traditional hallucination research focused on models generating falsehoods, we investigate the harm of misplaced correctness. We introduce a reproducible framework to elicit and measure HFH using controlled entity-level perturbations (both soft, embedding-based and hard, instruction-based) paired with strategic entity selection. Across summarization, rephrasing, and QA tasks, our evaluation of diverse LLMs reveals that HFH is a prevalent behavior that worsens with model scale. We identify three underlying mechanisms and demonstrate that a simple instructional prompt can reduce HFH rates by approximately 50%. Our framework turns the abstract factuality–faithfulness tension into a measurable, actionable target for building more reliable LLM systems. Our code is publicly available at https://github.com/ResponsibleAILab/Harmful-Factuality-Hallucination.
H. Zhang, H. Fan, K. Sha, Y. Huang, and Y. Feng.
DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration.
Findings of the Annual Meeting of the Association for Computational Linguistics (ACL Findings), 2025.
@inproceedings{zhang-etal-2025-dam,
title = "{DAM}: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration",
author = "Zhang, Hanzhi and
Fan, Heng and
Sha, Kewei and
Huang, Yan and
Feng, Yunhe",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2025",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-acl.242/",
doi = "10.18653/v1/2025.findings-acl.242",
pages = "4663--4676",
ISBN = "979-8-89176-256-5",
abstract = "Long-context understanding is crucial for many NLP applications, yet transformers struggle with efficiency due to the quadratic complexity of self-attention. Sparse attention methods alleviate this cost but often impose static, predefined masks, failing to capture heterogeneous attention patterns. This results in suboptimal token interactions, limiting adaptability and retrieval accuracy in long-sequence tasks. This work introduces a dynamic sparse attention mechanism that assigns adaptive masks at the attention-map level, preserving heterogeneous patterns across layers and heads. Unlike existing approaches, our method eliminates the need for fine-tuning and predefined mask structures while maintaining computational efficiency. By learning context-aware attention structures, it achieves high alignment with full-attention models, ensuring minimal performance degradation while reducing memory and compute overhead. This approach provides a scalable alternative to full attention, enabling the practical deployment of large-scale Large Language Models (LLMs) without sacrificing retrieval performance. DAM is available at: \url{https://github.com/HanzhiZhang-Ulrica/DAM}."
}
Long-context understanding is crucial for many NLP applications, yet transformers struggle with efficiency due to the quadratic complexity of self-attention. Sparse attention methods alleviate this cost but often impose static, predefined masks, failing to capture heterogeneous attention patterns. This results in suboptimal token interactions, limiting adaptability and retrieval accuracy in long-sequence tasks. This work introduces a dynamic sparse attention mechanism that assigns adaptive masks at the attention-map level, preserving heterogeneous patterns across layers and heads. Unlike existing approaches, our method eliminates the need for fine-tuning and predefined mask structures while maintaining computational efficiency. By learning context-aware attention structures, it achieves high alignment with full-attention models, ensuring minimal performance degradation while reducing memory and compute overhead. This approach provides a scalable alternative to full attention, enabling the practical deployment of large-scale Large Language Models (LLMs) without sacrificing retrieval performance. DAM is available at: https://github.com/HanzhiZhang-Ulrica/DAM.
S. Anjum, H. Zhang, W. Zhou, E. J. Paek, X. Zhao, and Y. Feng.
HALO: Hallucination Analysis and Learning Optimization to Empower LLMs with Retrieval-Augmented Context for Guided Clinical Decision Making.
IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (IEEE/ACM CHASE), 2025.
@inproceedings{10.1145/3721201.3721385,
author = {Anjum, Sumera and Zhang, Hanzhi and Zhou, Wenjun and Paek, Eun Jin and Zhao, Xiaopeng and Feng, Yunhe},
title = {HALO: Hallucination Analysis and Learning Optimization to Empower LLMs with Retrieval-Augmented Context for Guided Clinical Decision Making},
year = {2025},
isbn = {9798400715396},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3721201.3721385},
doi = {10.1145/3721201.3721385},
abstract = {Large language models (LLMs) have significantly advanced natural language processing tasks, yet they are susceptible to generating inaccurate or unreliable responses, a phenomenon known as hallucination. In critical domains such as health and medicine, these hallucinations can pose serious risks. This paper introduces HALO, a novel framework designed to enhance the accuracy and reliability of medical question-answering (QA) systems by focusing on the detection and mitigation of hallucinations. Our approach generates multiple variations of a given query using LLMs and retrieves relevant information from external open knowledge bases to enrich the context. We utilize maximum marginal relevance scoring to prioritize the retrieved context, which is then provided to LLMs for answer generation, thereby reducing the risk of hallucinations. The integration of LangChain further streamlines this process, resulting in a notable and robust increase in the accuracy of both open-source and commercial LLMs, such as Llama-3.1 (from 44\% to 66\%) and ChatGPT (from 56\% to 70\%). This framework underscores the critical importance of addressing hallucinations in medical QA systems, ultimately improving clinical decision-making and patient care. The open-source HALO is available at: https://github.com/ResponsibleAILab/HALO.},
booktitle = {Proceedings of the ACM/IEEE International Conference on Connected Health: Applications, Systems and Engineering Technologies},
pages = {187–198},
numpages = {12},
keywords = {large language models (LLMs), AI hallucination, retrieval augmented generation (RAG), chain of thought (CoT), few-shot question answering, clinical decision making},
location = {New York, NY, USA},
series = {CHASE '25}
}
Large language models (LLMs) have significantly advanced natural language processing tasks, yet they are susceptible to generating inaccurate or unreliable responses, a phenomenon known as hallucination. In critical domains such as health and medicine, these hallucinations can pose serious risks. This paper introduces HALO, a novel framework designed to enhance the accuracy and reliability of medical question-answering (QA) systems by focusing on the detection and mitigation of hallucinations. Our approach generates multiple variations of a given query using LLMs and retrieves relevant information from external open knowledge bases to enrich the context. We utilize maximum marginal relevance scoring to prioritize the retrieved context, which is then provided to LLMs for answer generation, thereby reducing the risk of hallucinations. The integration of LangChain further streamlines this process, resulting in a notable and robust increase in the accuracy of both open-source and commercial LLMs, such as Llama-3.1 (from 44% to 66%) and ChatGPT (from 56% to 70%). This framework underscores the critical importance of addressing hallucinations in medical QA systems, ultimately improving clinical decision-making and patient care. The open-source HALO is available at: https://github.com/ResponsibleAILab/HALO.
H. Zhang, H. Lucero, K. Sha, H. Fan, S. Fu, and Y. Feng.
D2SO: Detecting Distant and Small Objects for Vision-Based Vehicle Autonomous Systems.
IEEE International Conference on Mobility: Operations, Services, and Technologies (IEEE MOST), 2025.
@INPROCEEDINGS{11071446,
author={Zhang, Hanzhi and Lucero, Harold and Sha, Kewei and Fan, Heng and Fu, Song and Feng, Yunhe},
booktitle={2025 IEEE 3rd International Conference on Mobility, Operations, Services and Technologies (MOST)},
title={D2SO: Detecting Distant and Small Objects for Vision-Based Vehicle Autonomous Systems},
year={2025},
volume={},
number={},
pages={35-42},
keywords={YOLO;Image segmentation;Computer vision;Visualization;Autonomous systems;Image color analysis;Urban areas;Transformers;Hazards;Reliability;Small Object Detection;Distant Object Detection;Image Segmentation;Vehicle Autonomous Systems;Segment Anything Model (SAM);Vision Transformer (ViT)},
doi={10.1109/MOST65065.2025.00013}}
Detecting distant and small objects is a critical capability for vision-based vehicle autonomous systems, particularly in safety-critical scenarios such as assisted driving, where constant alertness, early reaction, and safe operation are very important. However, accurately recognizing tiny objects at a distance presents significant challenges due to limited pixel information, preprocessing-induced downscaling, and restricted detector resolution. To address these issues, this paper introduces D2SO, a Vision Transformer (ViT)-based framework specifically designed to enhance the detection of distant and small objects for autonomous systems. D2SO integrates multiple fine-tuned AI models, derived from the open-source Segment Anything Model (SAM), to detect distant objects such as static structures, humans, and vehicles, thereby improving situational awareness. The system employs visual cues through color mask overlays to efficiently convey essential information, ensuring users remain well-informed about detection outcomes. D2SO is explicitly optimized to detect distant entities that occupy fewer than 24 × 24 pixels on display, a scale often imperceptible to humans. Experimental results demonstrate that D2SO significantly outperforms baseline models, including SegFormer, YOLO v11 Segmentation, and U-Net, on a real-world street scene dataset spanning 50 cities, establishing its effectiveness in enhancing autonomous system performance.
T. Xie, H. Zhang, S. Wang, Y. Wan, I. Razzak, C. Kit, W. Zhang, and B. Hoex.
ByteScience: Bridging Unstructured Scientific Literature and Structured Data with Auto Fine-Tuned Large Language Model in Token Granularity.
IEEE International Conference on Data Mining Workshops (IEEE ICDMW), 2024.
@INPROCEEDINGS{10917307,
author={Xie, Tong and Zhang, Hanzhi and Wang, Shaozhou and Wan, Yuwei and Razzak, Imran and Kit, Chunyu and Zhang, Wenjie and Hoex, Bram},
booktitle={2024 IEEE International Conference on Data Mining Workshops (ICDMW)},
title={ByteScience: Bridging Unstructured Scientific Literature and Structured Data with Auto Fine-tuned Large Language Model in Token Granularity},
year={2024},
volume={},
number={},
pages={907-911},
keywords={Cloud computing;Accuracy;Web services;Large language models;Computational modeling;Propulsion;Natural language processing;Data models;Data mining;Informatics;component;formatting;style;styling;insert},
doi={10.1109/ICDMW65004.2024.00126}}
Natural Language Processing (NLP) is widely used to supply summarization ability from long context to structured information. However, extracting structured knowledge from scientific text by NLP models remains a challenge because of its domain-specific nature to complex data preprocessing and the granularity of multi-layered device-level information. To address this, we introduce ByteScience, a non-profit cloud-based auto fine-tuned Large Language Model (LLM) platform, which is designed to extract structured scientific data and synthesize new scientific knowledge from vast scientific corpora. The platform capitalizes on DARWIN, an open-source, fine-tuned LLM dedicated to natural science. The platform was built on Amazon Web Services (AWS) and provides an automated, user-friendly workflow for custom model development and data extraction. The platform achieves remarkable accuracy with only a small amount of well-annotated articles. This innovative tool streamlines the transition from the science literature to structured knowledge and data and benefits the advancements in natural informatics.