翻译:
How is AI (Artificial intelligence) data collected?
1. AI (Artificial intelligence) data processing
人工智能数据的采集是指按结构化、半结构化和非结构化三类形态获取训练所需样本的过程,来源上包括传感器数据、网络数据、企业内部数据与政府等机构发布的外部数据,解决的是模型训练所需数据的数量、类型与来源覆盖问题,并不解决数据本身的质量缺陷、隐私合规与标注偏差问题。
(2) Semi-structured data: Semi-structured data refers to the data that does not use the standard data model but has certain structured elements. For example, data formats such as XML files and HTML documents have certain structures and tags that can be analyzed by parsing and processing tools.
(3) Unstructured data: Unstructured data refers to text and media files that lack a clear structure, such as emails, videos, audio, images, and social media posts. This data is usually not suitable for storage in standard structured databases and requires special processing techniques.
2. Sources of AI (Artificial intelligence) data
(1) Sensor data: Sensor data usually comes from iot devices, such as temperature, humidity, acceleration, pressure, and so on. This data can be used to monitor the physical environment and system performance, and can be analyzed and predicted through machine learning applications.
(2) Network data: Network data refers to information publicly available through the Internet. For example, content such as social media, blogs, news articles, and emails are often extracted using Web crawlers or apis.
(3) Enterprise data: Enterprise data includes various types of structured and semi-structured data, such as transaction records, customer information and product information. This data is usually stored in enterprise databases and can be queried and analyzed through SQL or other technologies.
3. External data of AI (artificial intelligence)
External data refers to data sets that are not associated with a business or organization. For example, government data, meteorological data, geographic data, demographic data, and so on. This data is usually published by governments or other agencies and can be used to improve model accuracy or identify new business opportunities.
In general, the types and sources of data are diverse, and the application of artificial intelligence (AI) needs to select appropriate data collection and processing methods according to different data characteristics in order to obtain the best results when building models.