Improving safety in complex construction environments
Construction industry operations are heavily influenced by unpredictable factors such as weather, geological conditions, and constantly changing work environments, resulting in frequent hazardous incidents.
To enhance safety, there is growing demand for new support systems that leverage digital technologies.
One such demand is for a system capable of automatically extracting, visualising, and summarising potentially hazardous scenes from site-specific and time-dependent video footage and work logs.
However, within the construction industry, no technology is currently available for comprehensively supporting the entire process – from the accumulation of video data on construction machinery operations, to the extraction of risk scenes, the analysis and visualisation of near-miss incidents, and, ultimately, the generation of reports in an end-to-end manner.
Against this backdrop, SHI - leveraging its expertise in construction machinery and data analytics - and NEC, with its long-cultivated capabilities in video recognition and cutting-edge generative AI technologies, will jointly initiate the development of a first-of-its-kind system for the construction industry that automatically extracts near-miss incidents and generates corresponding reports based on camera footage and sensor data captured from hydraulic excavators.
From site data to actionable safety reports
The new system will utilise an extraction AI model, trained on real-world hydraulic excavator data accumulated on the SHI Group’s ICT/IoT common platform “SHICuTe”, to first identify and extract risk scenes from recorded video footage.
These risk scenes, together with operational data from the hydraulic excavators, will then be analysed using NEC’s proprietary technology, which combines video recognition with generative AI, and stored as multimodal data that incorporates temporal and spatial information.
Based on this data, along with SHI’s expertise in construction-site machinery operations and human workflows, the system will cross-reference hazardous and prohibited behaviour data, which are defined by accidents, construction equipment failures, and operations requiring particular attention, as well as company-specific data.
Based on these matching results, the system will automatically identify the risk scenes to report and generate high-quality near-miss reports that provide a concise summary of the circumstances surrounding each incident.
Prior to this initiative, a technical proof of concept was conducted in September 2025 to verify a system that automatically extracts near-miss incidents and generates reports from video footage captured by cameras mounted on hydraulic excavators.
The results confirmed that, based on the risk scenes extracted from the footage, the system reported near-miss cases, including potential accident scenarios and their associated circumstances.
Building on these outcomes, the joint development starting in April 2026 will aim to further expand the types of near-miss incidents that can be identified and enhance the report generation capabilities in line with customers’ safety management needs, thereby contributing to the realisation of safer construction sites.
Next steps and future development
In 2026, the system's technical development and validation will be advanced by utilising on-site data and safety management expertise from SHI, together with AI technologies and advanced technology consulting services from NEC, with the aim of achieving practical implementation in fiscal year 2027.
Looking ahead, the companies plan to broaden the system’s applicability beyond scenes where physical contact between workers and machinery may lead to occupational accidents to include unsafe conditions that may not be readily recognised by workers, as well as considerations for site-specific operational rules, thereby further expanding its scope of use.
SHI and NEC will continue to combine their respective strengths to develop and validate new technologies, contributing to the realisation of safer construction sites.