API reference¶

Generated from the docstrings. If something here is wrong or missing, fix the docstring rather than this page.

ents.dirtviz¶

Talking to the DirtViz backend.

Client interface with dirtviz.

TODO: - Add caching of data

class ents.dirtviz.client.Cell(data: str)[source]¶

Bases: object

Class representing a cell in the Dirtviz API.

class ents.dirtviz.client.BackendClient(base_url: str = 'https://dirtviz.jlab.ucsc.edu/api/')[source]¶

Bases: object

Client for interacting with the Dirtviz API.

get(endpoint: str, params: dict | None = None) → dict[source]¶

Get request to the API.

Parameters:

endpoint – The API endpoint to request.
params – Optional parameters for the request.

Returns:

A dictionary containing the response data.

static time_to_params(start: datetime, end: datetime) → dict[source]¶

Puts start and end datetime into an API paramter dictionary

Parameters:: dt – The datetime object to format.
Returns:: A string representing the formatted datetime.

power_data(cell: Cell, start: datetime, end: datetime) → DataFrame[source]¶

Gets power data for a specific cell by name.

Parameters:

cell – The Cell object for which to get power data.
start – The start date of the data.
end – The end date of the data.

Returns:

A pandas DataFrame containing the power data.

teros_data(cell: Cell, start: datetime, end: datetime) → DataFrame[source]¶

Gets teros data for a specific cell

Parameters:

cell – The Cell object for which to get teros data.
start – The start date of the data.
end – The end date of the data.

Returns:

A pandas DataFrame containing the teros data with columns vwc_raw, vwc_adj, temp, ec.

sensor_data(cell: Cell, name: str, meas: str, start: datetime, end: datetime, resample: str = 'none') → DataFrame[source]¶

Gets generic sensor data for a specific cell

Parameters:

cell – The Cell object for which to get sensor data.
name – Name of the sensor (e.g., “power”, “teros”).
meas – The measurement type (e.g., “v”, “i”, “vwc”, “temp”, “ec”).
start – The start date of the data.
end – The end date of the data.

Returns:

A pandas DataFrame containing the sensor data.

cell_from_id(cell_id: int) → Cell | None[source]¶

Get a Cell object from its ID.

Parameters:: cell_id – The ID of the cell.
Returns:: A Cell object. None if the cell does not exist.

cell_from_name(name: str) → Cell | None[source]¶

Get a Cell object from its name.

Parameters:: name – The name of the cell.
Returns:: A Cell object. None if the cell does not exist.

cells() → list[Cell][source]¶

Gets a list of all cells from the API.

Returns:: A list of Cell objects.

Common plots for data on dirtviz.

Still need to fill in with common functions: - Plot power (voltage, current, calcualted power) - Plot teros12 - Plot bme280 - Plot all (take all available sensors) - Plot group (find mean/stddev for each point)

Can pull from personal scripts from SenSys (jtmadden)

ents.dirtviz.plots.plot_data(data: list[DataFrame], name, **kwargs)[source]¶

Plots data from one or many cells

Parameters:

data – List of dataframes
name – Column or measurement name

ents.proto¶

Encoding and decoding the messages nodes upload.

Module for sensor measurements

Encode/decoding functions are wrappers around the protobuf messages. These take in the json dictionary format of the messages and return serialized byte arrays.

Format/parse functions implement the protocol for repeated sensor measurements. These take in a list of measurements and automatically optimize repeated metadata fields.

ents.proto.sensor.parse_sensor_measurement(data: bytes) → list[source]¶

Parses a sensor measurement into a usable dictionary.

Function does the following:

Decodes the serialized byte array
Updates metadata for each measurement if missing
Adds names, descriptions, and units to metadata

Parameters:: data – Byte array of message.
Returns:: Dictionary of sensor measurement.

ents.proto.sensor.format_sensor_measurement(meas: list) → bytes[source]¶

Formats a sensor measurement dictionary into a serialized byte array.

Function does the following:

Uses top level metadata for duplicate measurement metadata fields
Encodes the dictionary into a serialized byte array

Parameters:: meas – Dictionary of sensor measurement.
Returns:: Byte array of serialized message.

ents.proto.sensor.get_sensor_data(meas_type: int) → dict[source]¶

Gets sensor data information.

Parameters:: meas_type – Sensor measurement dictionary.
Returns:: Metadata associated with the sensor type.

ents.proto.sensor.encode_sensor_measurement(meas_dict: dict) → bytes[source]¶

ents.proto.sensor.encode_repeated_sensor_measurements(meas_dict: dict) → bytes[source]¶

Encodes a SensorMeasurement message

Parameters:: rep_meas – Repeated sensor measurement dictionary.
Returns:: Byte array of encoded RepeatedSensorMeasurements message.

ents.proto.sensor.decode_sensor_measurement(data: bytes) → dict[source]¶

Decodes a SensorMeasurement message

Parameters:: data – Byte array of SensorMeasurement message.
Returns:: Decoded sensor measurement dictionary.

ents.proto.sensor.decode_repeated_sensor_measurements(data: bytes) → dict[source]¶

Decodes repeated sensor measurements

Parameters:: data – Byte array from RepeatedSensorMeasurements
Returns:: List of decoded sensor measurement dictionaries.

ents.proto.sensor.update_repeated_metadata(meas: dict) → dict[source]¶

Ensures every measurements has metadata field set.

If a measurement is missing the metadata field, it is filled in from the repeated sensor measurement. Existing measurement metadata fields are not overwritten.

Parameters:: meas – Sensor measurement dictionary.
Returns:: Updated sensor measurement dictionary.

ents.proto.sensor.encode_sensor_response(resp_dict: dict) → bytes[source]¶

Encodes a sensor response message.

Shaped like:

{
    "responses": [
        {"status": int, "message": str},
        ...
    ]
}

Parameters:: resp_dict – Sensor response dictionary.
Returns:: Byte array of encoded SensorResponse message.

ents.proto.sensor.decode_sensor_response(data: bytes) → dict[source]¶

Decodes a sensor response message.

Shaped like:

{
    "responses": [
        {"status": int, "message": str},
        ...
    ]
}

Parameters:: data – Byte array of SensorResponse message.
Returns:: Decoded sensor response dictionary.

Module to encode soil power sensor protobuf messages

Encoding a response can be performed with

encode_response()

Each type of measurement has a corresponding encoding function as follows:

PowerMeasurement -> encode_power_measurement() Teros12Measurement -> encode_teros12_measurement()

ents.proto.encode.encode_response(success: bool = True) → bytes[source]¶

Encodes a Response message

The response indicates there was a successful upload, otherwise indicates an error.

Parameters:: success – Specifies whether to encode a success or failure.
Returns:: Byte string of response message.

ents.proto.encode.encode_power_measurement(ts: int, cell_id: int, logger_id: int, voltage: float, current: float) → bytes[source]¶

Encodes a PowerMeasurement within the Measurement message

Parameters:

ts – Timestamp in unix epochs
cell_id – Cell Id from Dirtviz
logger_id – Logger Id from Dirtviz
voltage – Voltage in V (Volts)
current – Current in A (Amps)

Returns:

Serialized Power measurement

ents.proto.encode.encode_teros12_measurement(ts: int, cell_id: int, logger_id: int, vwc_raw: float, vwc_adj: float, temp: float, ec: int) → bytes[source]¶

Encodes a Teros12Measurment within the Measurement message

Parameters:

ts – Timestamp in unix epochs
cell_id – Cell Id from Dirtviz
logger_id – Logger Id from Dirtviz
vwc_raw – Raw volumetric water content from Teros12
vwc_adj – Volumetric water content from Teros12 with calibration applied
temp – Temperature in C
ec – Electrical conductivity

Returns:

Serialized Teros12 measurement

ents.proto.encode.encode_phytos31_measurement(ts: int, cell_id: int, logger_id: int, voltage: float, leaf_wetness: float) → bytes[source]¶

Encodes a Phytos31Measurement within the Measurement message

Parameters:

ts – Timestamp in unix epochs
cell_id – Cell Id from Dirtviz
logger_id – Logger Id from Dirtviz
voltage – Raw voltage reading
leaf_wetness – Calibrated leaf wetness

Returns:

Serialized Phytos31 measurement

ents.proto.encode.encode_bme280_measurement(ts: int, cell_id: int, logger_id: int, pressure: int, temperature: int, humidity: int)[source]¶

Encodes a BME280Measurement within the Measurement message

The following raw values correspond to the following SI units

Raw

pressure: 98473 temperature: 2275 humidity: 43600

SI Units

pressure: 9847.3 hPa temperature: 22.75 C humidity: 43.600 %

Parameters:

ts – Timestamp in unix epochs
cell_id – Cell Id from Dirtviz
logger_id – Logger Id from Dirtviz
pressure – Ambient pressure
temperature – Ambient temperature
humidity – Relative humidity

Returns:

Serialized BME280 Measurement

ents.proto.encode.encode_teros21_measurement(ts: int, logger_id: int, cell_id: int, matric_pot: float, temp: float) → bytes[source]¶

ents.proto.encode.encode_user_configuration(logger_id: int, cell_id: int, Upload_method: str, Upload_interval: int, Enabled_sensors: list, Voltage_Slope: float, Voltage_Offset: float, Current_Slope: float, Current_Offset: float, WiFi_SSID: str, WiFi_Password: str, API_Endpoint_URL: str, API_Endpoint_Port: int) → bytes[source]¶

Encodes a UserConfiguration message

Parameters:

logger_id – ID of the logger.
cell_id – ID of the cell.
Upload_method – indicates whether LoRa or WiFi is used
Upload_interval – upload time in seconds
Enabled_sensors – used sensors
Voltage_Slope – Calibration slope for voltage.
Voltage_Offset – Calibration offset for voltage.
Current_Slope – Calibration slope for current.
Current_Offset – Calibration offset for current.
WiFi_SSID – WiFi SSID.
WiFi_Password – WiFi password.
API_Endpoint_URL
API_Endpoint_Port

Returns:

Serialized UserConfiguration message

Module to decode soil power sensor messages

ents.proto.decode.decode_response(data: bytes)[source]¶

Decodes a Response message.

Parameters:: data – Byte array of Response message.
Returns:: Returns the ResponseType.
Raises:: KeyError – Missing the resp field.

ents.proto.decode.decode_measurement(data: bytes, raw: bool = True) → dict[source]¶

Decodes a Measurement message

The data is decoded into a flat dictionary with the measurement type.

Parameters:

data – Byte array of Measurement message.
raw – Flag to return raw or adjusted measurements

Returns:

Flat dictionary of values from the meta field, measurement field, and the key “type” to indicate the type of measurement.

Raises:

KeyError – When the serialized data is missing a required field.

ents.proto.decode.decode_user_configuration(data: bytes) → dict[source]¶

Decodes a UserConfiguration message

Parameters:: data – Byte array of UserConfiguration message.
Returns:: Dictionary of UserConfiguration values.
Raises:: KeyError – When the serialized data is missing a required field.

ents.proto.esp32.encode_esp32command(cmd_type: str, **kwargs) → bytes[source]¶

Encodes a command for the esp32

Parameters:

cmd – Type of command to encode
**kwargs – See other encode functions

Returns:

Bytes of protobuf serialized message

ents.proto.esp32.encode_page_command(req: str, fd: int, bs: int = 0, n: int = 0) → <module 'google.protobuf.message' from '/opt/hostedtoolcache/Python/3.14.8/x64/lib/python3.14/site-packages/google/protobuf/message.py'>[source]¶

Encodes a command for memory paging

Parameters:

req – Page request type
fd – File descriptor
bs – Block size
n – Number of bytes

Returns:

PageCommand message

Raises:

NotImplementedError – When page request type does not exist

ents.proto.esp32.encode_test_command(state: str, data: int) → <module 'google.protobuf.message' from '/opt/hostedtoolcache/Python/3.14.8/x64/lib/python3.14/site-packages/google/protobuf/message.py'>[source]¶

Encodes a command for testing modules library

Valid strings for state:

receive: Data is received by the module
receive_request: Data is received by the module, indicating data for a subsequent request
request: Data is sent by the module

Parameters:

state (str) – State command string
data (int) – Integer to test passing of data

Returns:

TestCommand message

Raises:

NotImplementedError – When the ChangeState type does not exist

ents.proto.esp32.encode_wifi_command(_type: str, ssid: str = '', passwd: str = '', url: str = '', port: int = 0, rc: int = 0, ts: int = 0, resp: bytes = b'') → <module 'google.protobuf.message' from '/opt/hostedtoolcache/Python/3.14.8/x64/lib/python3.14/site-packages/google/protobuf/message.py'>[source]¶

Encodes a WiFi command

Valid strings for _type are:: “connect”: Connecting to a WiFi network and any other setup “post”: Posting data to the endpoint

Not all arguments are necessary for valid commands.

Parameters:

request (_type; Type of)
ssid – WiFi SSID
passwd – WiFi password
url – Endpoint URL
port – Port of webserver
rc – Return code
ts – Timestamp
resp – Binary data response from server

Returns:

WiFiCommand message

Raises:

NotImplementedError – When _type does not exist

ents.proto.esp32.decode_esp32command(data: bytes) → dict[source]¶

ents.config¶

Node user configuration.

@brief PyQt5 GUI Application for Configuring User Settings

This module provides a PyQt5-based graphical interface for configuring user settings. The application allows users to input configuration details, including Logger ID, Cell ID, Upload Method (WiFi or LoRa), Upload Interval, Enabled Sensors, and Calibration parameters for voltage and current (V/I Slope and Offset).

Key features:

Save and Load: Users can save configurations to a file or load previous configurations for easy reuse.
Real-time Configuration: By pressing the “Send Configuration” button, the settings are serialized with Protobuf and transmitted over UART to the STM32 for direct application.

@file user_config.py @author Ahmed Hassan Falah @date 2024-10-10

class ents.config.user_config.Ui_MainWindow[source]¶

Bases: object

setupUi(MainWindow)[source]¶

@brief Sets up the user interface components.

Initializes the main window and creates the layout for user configurations.

setupGroupBoxes()[source]¶: @brief Sets up the group boxes for different configuration sections.

toggleUploadMethod()[source]¶: @brief Shows or hides WiFi settings based on the upload method selected.

showWiFiSettings()[source]¶: @brief Displays the WiFi configuration settings.

hideWiFiSettings()[source]¶: @brief Hides the WiFi configuration settings.

createLabel(text, font)[source]¶

@brief Creates a label with the specified text and font.

@param text Text for the label. @param font Font settings for the label. @return QLabel instance

createLineEdit(placeholder)[source]¶

@brief Creates a QLineEdit with a placeholder.

@param placeholder Placeholder text for the QLineEdit. @return QLineEdit instance

setupSaveAndLoadButtons()[source]¶: @brief Creates and configures the save and load buttons.

saveConfiguration(flag: str)[source]¶: @brief Validates inputs, encodes the configuration, and sends it via UART.

loadConfiguration(flag: str)[source]¶: @brief Loads configuration from a selected JSON file and fills the input fields.

sendToUART(encoded_data)[source]¶

@brief Sends the encoded configuration data via UART.

@param data Encoded configuration data.

receiveFromUART()[source]¶

@brief Receives the current encoded configuration data via UART.

@param void. @return (success, data): A tuple containing success status and decoded data or an error message.

validateURL(value, field_name)[source]¶: @brief Validates that the input is a valid URL. @param value The input value to validate. @param field_name The name of the field being validated. @return The validated URL string. @throws ValueError if the input is invalid.

validateUInt(value, name)[source]¶

@brief Validates that the input is a positive integer.

@param value The input value to validate. @param name The name of the parameter (for error messages). @return Validated unsigned integer value.

validateInt(value, name)[source]¶

@brief Validates that the input is an integer.

@param value The input value to validate. @param name The name of the parameter (for error messages). @return Validated integer value.

validateFloat(value, name)[source]¶

@brief Validates that the input is a float.

@param value The input value to validate. @param name The name of the parameter (for error messages). @return Validated float value.

ents.config.user_config.main()[source]¶

ents.simulator¶

Standing in for hardware.

class ents.simulator.node.NodeSimulator(cell: int, logger: int, sensors: list[str], fn=<built-in function sin>)[source]¶

Bases: object

Simulation class to simulate measurements for different sensors

send_next(url: str) → bool[source]¶

Sends measurements to a dirtviz instance

Parameters:: url – URL of the dirtviz instance
Returns:: True if there are measurements to send, False otherwise

measure(ts: int)[source]¶

Simulate measurements

Parameters:: ts – Timestamp of the measurement
Returns:: None

class ents.simulator.node.NodeSimulatorGeneric(cell: int, logger: int, sensors: list[str], _min=-1, _max=1, fn=<built-in function sin>)[source]¶

Bases: object

Simulation class to simulate measurements for different sensors

send_next(url: str) → bool[source]¶

Sends measurements to a dirtviz instance

Parameters:: url – URL of the dirtviz instance
Returns:: True if there are measurements to send, False otherwise

measure(ts: int)[source]¶

Simulate measurements

Parameters:: ts – Timestamp of the measurement

last_measurement() → bytes[source]¶

Gets the last encoded measurement.

Returns:: Last encoded measurement.

last_request() → str[source]¶

Gets the last sent request.

Returns:: Formatted headers and body.

last_response() → str[source]¶

Gets the last response.

Returns:: Response from server.

ents.calibrate¶

Calibration against a reference instrument.

ents.calibrate.linear_regression.linear_regression(meas: list, actual: list)[source]¶

Performs linear regression between measured and actual values

Parameters:

meas – Measure values
actual – Ground truth values

Returns:

Linear regression model

ents.calibrate.linear_regression.print_eval(pred: list, actual: list)[source]¶

Prints various evaluation parameters of the model

The following metrics are used:

Mean absolute error
Root mean square error
R2 score
Mean absolute percentage error

Parameters:

pred – Predicted values
actual – Actual values

ents.calibrate.linear_regression.print_coef(model)[source]¶

Print coefficients of a model

Parameters:: model – Linear model

ents.calibrate.linear_regression.print_norm(residuals: list)[source]¶

Print mean offset and standard deviation

Parameters:: residuals – List of residuals

Data recorder for Soil Power Sensor

The recorder controls the SPS firmware and a Keithley 2400 Source Measurement Unit (SMU). Ideally the script should work with any microcontrollre flashed with the firmware and any source measurement unit that supports Standard Commands for Programable Instruments (SCPI). The units listed are the ones that the script was developed and tested on. It allows to step through a range of output voltages on the Keithley and measure the voltage and current from both the SMU and the Soil Power Sensor (SPS).

class ents.calibrate.recorder.SerialController(port, baudrate=115200, xonxoff=False)[source]¶

Bases: object

Generic serial controller that will open and close serial connections

ser = None¶

class ents.calibrate.recorder.LANController(host, port)[source]¶

Bases: object

Generic LAN controller that will open and close LAN connections

sock = None¶

class ents.calibrate.recorder.SoilPowerSensorController(port)[source]¶

Bases: SerialController

Controller used to read values from the SPS

get_power() → tuple[float, float][source]¶

Measure voltage from SPS

Returns¶

tuple[float, float: voltage, current

check()[source]¶

Performs a check of the connection to the board

Raises¶

RuntimeError: Checks that SPS replies “ok” when sent “check”

class ents.calibrate.recorder.SMUSerialController(port, source_mode)[source]¶

Bases: SerialController

Controller for the Keithley 2400 SMU used to supply known voltage to the SPS

Uses SCPI (Standard Control of Programmable Instruments) to control the SMU through its RS232 port. Written for the Keithley 2400 SMU, but should work for any other SMU that uses SCPI.

class VoltageIterator(ser, start, stop, step)[source]¶

Bases: object

VoltageIterator Class

Implements a iterator for looping through voltage output values

set_voltage(v)[source]¶: Sets the voltage output

vrange(start, stop, step) → VoltageIterator[source]¶

Gets iterator to range of voltages

Parameters¶

startfloat: Starting voltage
stopfloat: End voltage
stepfloat: Voltage step

get_voltage() → float[source]¶

Measure voltage supplied to the SPS from SMU

Returns¶

float: Measured voltage

get_current() → float[source]¶

Measure current supplied to the SPS from SMU

Returns¶

float: Measured current

class ents.calibrate.recorder.SMULANController(host, port)[source]¶

Bases: LANController

Controller for the Keithley 2400 SMU used to supply known voltage to the SPS

Uses SCPI (Standard Control of Programmable Instruments) to control the SMU through its RS232 port. Written for the Keithley 2400 SMU, but should work for any other SMU that uses SCPI.

class VoltageIterator(sock, start, stop, step)[source]¶

Bases: object

VoltageIterator Class

Implements a iterator for looping through voltage output values

set_voltage(v)[source]¶: Sets the voltage output

class CurrentIterator(sock, start, stop, step)[source]¶

Bases: object

CurrentIterator Class

Implements a iterator for looping through current output values

set_current(i)[source]¶: Sets the current output

setup_voltage()[source]¶: Configures smu for sourcing voltage

setup_current()[source]¶: Configured smu for sourcing current

vrange(start, stop, step) → VoltageIterator[source]¶

Gets iterator to range of voltages

Parameters¶

startfloat: Starting voltage
stopfloat: End voltage
stepfloat: Voltage step

irange(start, stop, step) → CurrentIterator[source]¶

Gets iterator to range of currents

Parameters¶

startfloat: Starting current
stopfloat: End current
stepfloat: Current step

get_voltage() → float[source]¶

Measure voltage supplied to the SPS from SMU

Returns¶

float: Measured voltage

get_current() → float[source]¶

Measure current supplied to the SPS from SMU

Returns¶

float: Measured current

class ents.calibrate.recorder.Recorder(serialport: str, host: str, port: int, delay: int = 1)[source]¶

Bases: object

record_voltage(start: float, stop: float, step: float, samples: int) → dict[str, list][source]¶

Records voltage measurements from the smu and sps

Input arguments given in volts units. Returned dictionary has keys for the expected voltage (number from python), actual voltage (value measured by the smu), and meas voltage (value measured by the sps).

Parameters:

start – Start voltage
stop – Stop voltage (inclusive)
step – Step between voltage
samples – Number of samples taken at each voltage

Returns:

Dictionary in a pandas compatible format:

{
    "expected": [],
    "actual": [],
    "meas": [],
}

record_current(start: float, stop: float, step: float, samples: int) → dict[str, list][source]¶

Records current measurements from the smu and sps

Input arguments given in amps units. Returned dictionary has keys for the expected current (number from python), actual current (value measured by the smu), and meas current (value measured by the sps).

Parameters:

start – Start current
stop – Stop current (inclusive)
step – Step between current
samples – Number of samples taken at each voltage

Returns:

Dictionary in a pandas compatible format:

{
    "expected": [],
    "actual": [],
    "meas": [],
}