Stream data processing refers to the technology to process stream data in real time. Stream data means the data that is created continuously and has the time stamp (information on the time when the data is created or updated), such as stock trading information and traffic information.
In the case of data processing using a database, data is entered and accumulated in the database and a request for certain data (called query) is issued at any time. For stream data processing, in contrast, the query is registered in advance so that the query is executed when the relevant data is entered. The entered data is processed in memory devices in order, instead of being stored in storage devices, etc., so that the data that arrives one after another is processed continuously. In this manner, real-time processing is achieved by eliminating the time lag from when the data is created.
Research on stream data processing started early in the 2000's. In Japan, it gathered attention in accordance with an increase in the data volume handled in business operations and daily lives, and products incorporating stream data processing functions were launched in the latter half of the 2000's. Going forward, the projected demand for processing data from sensors will grow even further as IoT applications will evolve, and stream data processing is expected to be further utilized.
