Cornell University · Cornell Smart Farm · M.Eng. Design Project · Embedded Systems & Product Design · 2024.08-2025.12 · Published: 2025-12-31
Intelligent Voice-Activated Data Collection System for Precision Agriculture
An offline-first, tractor-mounted platform for hands-free agricultural field data collection
01 · Project Information
Project Duration: August 2024 - December 2025
Project Context: Cornell Smart Farm
Project Type: Cornell University M.Eng. Design Project
Advisors: Prof. Joseph Skovira and Prof. Louis Longchamps
Role: Embedded Systems and Product Design
02 · Project Overview
This project involved the design and implementation of a tractor-mounted, voice-activated data collection system for precision agriculture.
During field operations, farm workers may need to drive machinery, inspect crops, and record observations simultaneously. Paper notes, smartphones, and handheld terminals can interrupt the workflow, especially when the user's hands are occupied or network connectivity is unavailable.
The system enabled workers to capture crop observations, equipment issues, field information, and follow-up tasks through natural voice interaction.
Each recording was associated with its time, location, and agricultural task. The system stored the original audio as an MP3 file and generated a corresponding JSON record. When network access became available, the records could be synchronized for full transcription, language processing, and agricultural data analysis.
03 · Project Background
Smart farming depends on high-quality field data, but agricultural environments introduce several practical constraints:
Workers frequently operate vehicles or equipment with both hands
Tractors produce continuous noise and vibration
Network connectivity may be unavailable in remote fields
Field observations are time-sensitive and location-specific
Records may be distributed across paper, phones, and human memory
Long recordings can be difficult to associate with individual fields and tasks
Vehicle-mounted equipment must operate from tractor power
The system therefore adopted an offline-first, voice-driven, location-aware, and structured-storage architecture.
04 · Project Objectives
The primary objectives were to:
Enable hands-free recording of crop and equipment observations
Reduce interaction complexity through voice activation
Support local data capture when network access was unavailable
Associate voice records with GPS locations
Segment long recordings according to tasks and field boundaries
Preserve both original audio and structured metadata
Perform complete transcription and analysis after connectivity returned
Develop hardware suitable for tractor installation and power
Display recording, location, and system status through a graphical interface
Improve the reliability of agricultural field-data collection
05 · User Scenario
The system was designed for farm workers driving tractors or performing field operations.
Typical use cases included:
Recording crop-growth anomalies
Reporting pest or irrigation problems
Documenting equipment failures
Marking fields requiring additional inspection
Recording fertilizer, seed, or supply requirements
Separating records across different fields
Preserving observations without network access
The system was not designed to replace agricultural expertise. Its purpose was to reduce the effort required to convert field observations into digital records.
06 · System Architecture
The system consisted of:
Voice activation and interaction
Microphone and audio acquisition
Offline keyword recognition
Agricultural task classification
GPS positioning and field-boundary detection
Local data storage
MP3 audio management
JSON metadata generation
A 7-inch graphical user interface
Network recovery and data synchronization
Raspberry Pi embedded computing
12V-to-5V power conversion
A 3D-printed enclosure
The architecture connected voice input, task confirmation, data collection, location association, local storage, and later analysis within an end-to-end workflow.
07 · System Workflow
The system followed the process below:
The worker spoke an activation phrase such as “Hi Tractor”
The system detected the phrase and initiated the interaction
The system asked the user to select a task type
The user selected or described the task through speech
Audio recording began and a timestamp was created
The GPS module continuously recorded the location
The recording was associated with its task and geographic position
A field-boundary transition triggered a confirmation prompt
The user could end the current segment or begin a new one
The Raspberry Pi saved the original audio as an MP3 file
A corresponding JSON record was generated
Offline speech processing recognized wake words and essential commands
When connectivity returned, records were synchronized for full transcription
The backend could perform classification and agricultural analysis
08 · Voice Interaction
Voice Activation
A voice-activation mechanism reduced the need for touch interaction.
An example activation phrase was:
Hi Tractor
After activation, the system entered task-confirmation mode and prompted the worker through the display or voice interface.
This interaction allowed workers to continue operating agricultural equipment without using a handheld device.
Task Classification
Before recording, the system identified the current agricultural task.
Task categories could include:
Crop observations
Pest or disease reports
Irrigation problems
Equipment failures
Maintenance tasks
Field inspections
Agricultural supply requirements
Other field notes
The task label was stored together with the audio, timestamp, and GPS information.
Long-Recording Segmentation
Long recordings could be segmented according to task changes and field position.
Each segment could be associated with:
Task category
Start and end time
Starting position
Ending position
Field information
Audio file path
Transcription status
Synchronization status
This structure prevented an entire field operation from being stored as one difficult-to-search audio recording.
09 · GPS and Field-Boundary Detection
The GPS module recorded the worker's location and helped identify transitions between fields.
When the system detected that the user was leaving one field or entering another, it could ask whether to:
End the current task record
Continue the same task
Start a new recording segment
Associate subsequent content with the new field
Combining voice records with GPS information provided a spatial context for agricultural observations and supported later precision-agriculture analysis.
10 · Offline-First Architecture
Agricultural fields may have limited or unstable connectivity. The system was therefore designed to remain functional without a network connection.
Offline functions included:
Voice activation
Essential command recognition
Audio acquisition
Task classification
GPS data collection
MP3 file storage
JSON record generation
Local record browsing
Synchronization queuing
Full speech transcription and more advanced language processing could be performed after network access returned.
This approach prevented network interruptions from making the system unusable.
11 · Data Storage
MP3 Audio
The original voice observations were stored as MP3 files.
Preserving the raw audio enabled:
Future retranscription
Retention of context and tone
Manual verification
Reprocessing with improved models
Recovery from early recognition errors
JSON Metadata
Each audio recording was associated with a JSON record.
The structured data could include:
Record identifier
Task category
Start and end time
GPS coordinates
Field identifier
Audio filename
Local file path
Keywords
Transcript
Transcription status
Upload status
User confirmation status
Associating MP3 and JSON files preserved the original evidence while supporting structured retrieval and analysis.
7-Inch Graphical Interface
A 7-inch display provided clear feedback about the operating status.
The interface displayed:
Current system state
Recording status
Current task category
Recording duration
GPS connection
Location information
Network status
Local storage status
Synchronization progress
Saved records
Errors and user prompts
The interface emphasized large information elements and simple interactions suitable for a tractor environment.
13 · Hardware Design
Raspberry Pi Platform
The Raspberry Pi served as the primary processing unit and was responsible for:
Running the Linux operating system
Receiving microphone input
Performing offline keyword recognition
Reading GPS data
Managing local files
Generating JSON records
Driving the graphical interface
Managing connectivity
Synchronizing data
Vehicle Power
A tractor can provide a 12V electrical supply, while the Raspberry Pi and peripherals require a stable 5V input.
A 12V-to-5V DC-DC power-conversion system was developed for the Raspberry Pi, display, and connected devices.
The power design considered:
Output-voltage stability
Raspberry Pi peak demand
Display and peripheral consumption
Voltage fluctuations during vehicle operation
Connection reliability
Continuous operation
3D-Printed Enclosure
A 3D-printed enclosure was designed and manufactured to integrate and protect the system.
The enclosure accommodated:
Raspberry Pi
Display
GPS module
Microphone and cables
Power-conversion module
Data and power interfaces
The mechanical design also considered installation, cooling, cable organization, usability, and maintenance.
14 · Synchronization and Analysis
When connectivity returned, the system could synchronize locally stored records with a cloud service.
The post-processing workflow could support:
Full speech transcription
Agricultural keyword extraction
Task classification
Field association
Anomaly summarization
Farm work-log generation
Equipment-maintenance records
Agricultural supply identification
Procurement lists were treated as an optional future output. The system could identify potential supply requirements and generate a list for human confirmation, rather than executing purchases automatically.
15 · System Testing
Testing covered voice interaction, GPS acquisition, local storage, graphical status display, power delivery, and the complete end-to-end workflow.
The tests included:
Wake-word and keyword recognition
Audio recording under different noise conditions
Long-duration recording
Matching task labels with audio files
GPS data acquisition
Field-transition prompts
MP3 and JSON generation
Local storage during network outages
Synchronization after connectivity returned
Status presentation on the 7-inch interface
Stability of the 12V-to-5V power supply
Continuous system operation
Enclosure installation and component protection
Hardware, power, storage, and software optimizations improved overall system reliability by 40%.
16 · Challenges and Solutions
17 · My Responsibilities
My responsibilities included:
Analyzing field-data collection requirements
Designing the end-to-end workflow and voice interaction
Building the Raspberry Pi embedded system
Designing voice activation and task-classification logic
Developing audio acquisition and local storage
Defining the MP3 and JSON data structure
Integrating GPS and field-boundary prompts
Developing the 7-inch graphical interface
Designing the 12V-to-5V vehicle power system
Supporting 3D-printed enclosure design and assembly
Testing offline operation and later synchronization
Improving system stability and usability
Preparing project documentation and demonstrations
18 · Key Contributions and Outcomes
Developed a tractor-mounted agricultural voice data collection system
Implemented hands-free voice activation and task confirmation
Supported data collection without network connectivity
Implemented offline recognition for wake words and essential commands
Preserved original recordings as MP3 files
Stored time, task, and location metadata in JSON
Integrated GPS positioning and field-boundary prompts
Segmented long recordings by task and location
Developed a 7-inch embedded graphical interface
Designed a 12V-to-5V vehicle power system
Built a 3D-printed enclosure for system integration
Improved overall system reliability by 40%
Established an end-to-end workflow from field capture to cloud analysis
19 · Project Impact
The system transformed fragmented, memory-dependent field observations into structured agricultural data.
Its primary value included:
Reducing manual interaction during tractor operation
Decreasing dependence on continuous network connectivity
Preserving original field observations
Associating voice information with tasks and locations
Improving data completeness and traceability
Providing a data foundation for precision agriculture
Supporting farm management and maintenance planning
20 · Conclusion
This project integrated Raspberry Pi, voice interaction, GPS positioning, local storage, a graphical interface, vehicle power conversion, and cloud analysis into a unified smart-farm data collection platform.
Through the project, I strengthened my skills in embedded Linux, speech-data processing, GPS integration, structured data design, graphical interface development, power-system design, and physical prototyping.
More importantly, the project taught me how to design an end-to-end embedded product for a constrained real-world environment involving limited connectivity, background noise, vehicle power, and hands-free interaction.