Data Collection and Processing Based on Python and Wi-Fi

01 · Project Information

• Project Duration: April 2021 - May 2022

• Role: Team Leader

• Advisor: Prof. Yongzhi Jing, Southwest Jiaotong University

02 · Project Overview

This project focused on data collection and processing using Python and Wi-Fi communication, aiming to achieve highly reliable real-time data transmission and advanced speech signal processing.

The project covered three primary areas:

• Developing a Python-based TCP interactive system for real-time data collection

• Improving transmission reliability between senders and receivers

• Applying deep learning-based signal processing to improve speech clarity

The project was completed 20% ahead of schedule while maintaining 100% adherence to progress milestones.

03 · Project Objectives

The primary objectives were to:

1. Develop a highly reliable Wi-Fi-based real-time data collection system

2. Establish robust TCP communication to prevent data loss

3. Improve speech signal clarity using deep clustering

4. Optimize the system for future IoT and real-time monitoring applications

04 · Technical Implementation

1. Python-Based TCP Interactive Data Collection System

A custom Python-based TCP communication framework was developed for real-time data collection and transmission.

The implementation included:

• Establishing TCP connections between senders and receivers

• Designing a real-time data transmission and interaction mechanism

• Implementing error detection and correction

• Optimizing TCP communication parameters

• Verifying the integrity of received data

• Recording system transmission status

The system achieved 99.5% data transmission reliability between senders and receivers.

2. Deep Clustering for Speech Signal Processing

Deep clustering was applied to separate overlapping speech signals and improve speech clarity.

The signal-processing workflow included:

• Preprocessing the original speech signals

• Converting speech signals into the frequency domain

• Extracting relevant speech frequency components

• Applying deep clustering to distinguish overlapping voices

• Reconstructing the separated speech signals

• Comparing speech clarity before and after processing

The combination of deep clustering and frequency-domain analysis improved overlapping speech-signal separation by 30%.

3. System Performance Optimization

The system was optimized for transmission latency, real-time processing efficiency, and scalability.

The optimization work included:

• Conducting tests in real-world environments

• Optimizing the data transmission and reception process

• Improving buffer-management strategies

• Applying Python multithreading to improve processing efficiency

• Reducing waiting time during data transmission and processing

• Designing a modular software architecture

The modular architecture made the system adaptable to larger datasets and future IoT monitoring applications.

05 · System Workflow

The system followed the process below:

1. Collect raw data at the sender

2. Format the collected data

3. Establish a TCP connection through Wi-Fi

4. Transmit the data to the receiver in real time

5. Verify the integrity of the transmitted data

6. Perform frequency-domain analysis on speech data

7. Apply deep clustering to separate overlapping speech signals

8. Output and store the processed data

9. Record system status and test results

06 · Challenges and Solutions

07 · My Responsibilities

As the team leader, I was responsible for:

• Defining the research objectives and implementation plan

• Dividing the project into milestones and tracking progress

• Coordinating tasks among team members

• Participating in the design of the Python TCP communication system

• Managing data-transmission reliability testing

• Supporting the speech-signal processing research

• Organizing system integration and performance evaluation

• Preparing project documentation and results

• Managing project progress and milestone delivery

Through structured task allocation and progress management, the team completed the project 20% ahead of schedule while meeting 100% of its planned milestones.

08 · Key Contributions and Outcomes

• Developed a real-time data collection system using Python and Wi-Fi

• Established an interactive TCP communication mechanism

• Achieved 99.5% data transmission reliability

• Applied deep clustering to overlapping speech signals

• Improved speech-signal separation by 30%

• Improved real-time processing through multithreading and buffer optimization

• Designed a modular architecture for future system expansion

• Completed the project 20% ahead of schedule

• Maintained 100% adherence to project milestones

09 · Project Impact

The project demonstrated the feasibility of using Python and TCP communication for real-time wireless data collection. It also integrated data transmission and speech-signal processing within a unified system.

The modular architecture could be extended to:

• IoT data acquisition

• Environmental monitoring

• Real-time equipment monitoring

• Wireless sensor data transmission

• Intelligent speech processing

• Mobile data collection

10 · Conclusion

This project successfully implemented a Python-based TCP interactive data collection system with highly reliable Wi-Fi transmission and advanced speech-signal processing capabilities.

Through the project, I strengthened my knowledge of wireless communication, TCP data transmission, Python multithreading, buffer management, and speech-signal processing. I also gained practical experience in team leadership, project planning, system testing, and performance optimization.

11 · Future Work

Future development could focus on:

• Integrating AI-driven adaptive learning to improve real-time speech-processing accuracy

• Expanding the system into broader IoT applications such as environmental monitoring

• Optimizing wireless communication protocols to reduce latency in large-scale deployments

• Developing a mobile-friendly version for portable real-time data collection