Package 'erdatools'

Description

Starts the plumber REST API bundled with erdatools. The API provides endpoints for ERDA directory indexing, sourceimages pipelines, and deployment management. Opens a Swagger UI in the browser by default.

Usage

et_api_run(port = 8000, host = "127.0.0.1", ...)

Arguments

Value

Invisibly returns the plumber router (after the server stops).

Description

Transforms a filtered ERDA index (files only) into the shape needed to write CamAlien on-board images into the camalien database data2 model. Every image attribute is parsed from the filename via et_parse_camalien_filename() — no EXIF reads are performed. One row is produced per image file.

Usage

et_build_camalien_images(index, sessionid)

Arguments

Details

Filename tokens map to columns as follows: GT -> timestamp, CT -> computertime, lon/lat -> point position, expt -> exposuretime, gain, alt -> altitude, vel -> velocity, tag. CT is the onboard computer clock and GT the GPS-derived time; the backend treats the GPS time as the authoritative capture timestamp and keeps the computer clock in computertime (verified against the live camalien DB). The timestamps are UTC wall-clock in ISO 8601 basic format (e.g. ⁠20250403T125240.225632⁠) and are stamped with a trailing Z so the stored instant is independent of the database session timezone. Tokens absent from a filename become NA. All rows are stamped status = 1 (the initial data2.imagedata_status, named pending); the backend PlantNet pipeline advances the status later.

Duplicate filenames in the index (the same file listed in more than one directory, e.g. a folder copied into a sibling on ERDA) are dropped with a warning, keeping the first occurrence. data2.imagefile.filename is globally unique, so a repeated filename is the same image; writing both would insert an orphaned data2.imagedata row whose imagefile insert is silently discarded by ⁠ON CONFLICT DO NOTHING⁠.

Value

Data frame with one row per image and columns filename, path, type, sessionid, lon, lat, timestamp, status, tag, computertime, exposuretime, gain, altitude, velocity.

Examples

## Not run: 
idx <- et_index_filter("erda_index.parquet", extensions = c("jpg", "tiff"))
imgs <- et_build_camalien_images(idx, sessionid = "3f2c8b1e-...-...")

## End(Not run)

Description

Produces up to two data.classifications rows per edge detection: an order classification (from the ⁠order_*⁠ fields) and a species classification (from the ⁠species_*⁠ fields). Detections are matched to their database UUIDs by joining on key, so parsed_detect and detection_ids must come from the same session (the edge key is unique per session only).

Usage

et_build_classifications(
  parsed_detect,
  detection_ids,
  order_algorithm = "mcc24",
  species_algorithm = "uk-denmark-moths"
)

Arguments

Details

Rows whose label is NA/empty are skipped. Following the database's own backfill (migration 2044), the single edge aboveTH flag is the order threshold, so the order row carries it as abovethreshold and the species row's abovethreshold is NA (SQL NULL).

Value

A data frame with columns detectionid, taxonlevel, label, labelid, score, abovethreshold, algorithm.

Examples

## Not run: 
parsed <- et_parse_edge_detections("results/detect/20250508.csv")
det_ids <- et_db_write_detections(con, et_build_detections(parsed, si_ids))
classifications <- et_build_classifications(parsed, det_ids)

## End(Not run)

Description

Reads a CSV file of deployment records and returns a tibble ready for et_db_write_deployments(). Validates required columns, normalizes values, and optionally resolves partner names to IDs.

Usage

et_build_deployments(csv_path, projectid, partner_map = NULL)

Arguments

Details

The Partner column must match values in data.partners.name exactly.

Required CSV columns: Code, Partner, ⁠Location name⁠, Year, Latitude, Longitude.

Optional CSV columns (filled with NA if absent): Elevation (m), ⁠Solar or grid power⁠, ⁠Trap serial number⁠, ⁠Trap type⁠, Comments.

Value

A tibble::tibble() with columns year, code, locationname, elevation, latitude, longitude, power, trapserialno, trapstatus, traptype, comment, projectid, and either partnerid (when partner_map is supplied) or partner (when it is not).

Examples

## Not run: 
# With partner resolution
partners <- DBI::dbGetQuery(con, "SELECT id, name FROM data.partners")
pmap <- setNames(partners$id, partners$name)
df <- et_build_deployments("deployments.csv", "ias", partner_map = pmap)

# Without (caller resolves later)
df <- et_build_deployments("deployments.csv", "ias")

## End(Not run)

Description

Joins parsed track-detail rows (which map detection key to a track edge_id) to detection UUIDs (by key) and track UUIDs (by edge_id), producing data.detection_tracks rows. Detections whose key is absent from the track-detail file, or whose track has no UUID, produce no row.

Usage

et_build_detection_tracks(
  trs_parsed,
  detection_ids,
  track_ids,
  algorithm = "tracy",
  algorithmversion = "luke"
)

Arguments

Details

algorithm and algorithmversion are denormalised onto every junction row and must equal the referenced track's identity — the composite foreign key ⁠(trackid, algorithm, algorithmversion)⁠ -> data.tracks fails otherwise. Only tracks whose identity matches these arguments are considered when resolving trackid.

Value

A data frame with columns detectionid, trackid, algorithm, algorithmversion.

Examples

## Not run: 
trs <- et_parse_edge_track_detections("results/tracks/20250508TRS.csv")
track_ids <- et_db_get_tracks(con, sessionid)
links <- et_build_detection_tracks(trs, det_ids, track_ids)

## End(Not run)

Description

Transforms the output of et_parse_edge_detections() into the data.detections schema. Each edge detection becomes one row: the source image is resolved by joining the detection filename (as a basename) to sourceimage_ids, and the bounding box is rounded to integer pixels. The contour, cropurl, and contour_geom columns are left NULL (the edge pipeline provides no polygon). key and detect_id are carried through for the downstream classifications and detection-track joins.

Usage

et_build_detections(parsed_detect, sourceimage_ids)

Arguments

Details

Detections whose filename has no matching source image are dropped (a warning reports the count).

Value

A data frame with columns sourceimageid, bbox_x1, bbox_y1, bbox_x2, bbox_y2 (integer), plus key and detect_id for downstream joins.

Examples

## Not run: 
parsed <- et_parse_edge_detections("results/detect/20250508.csv")
si_ids <- et_db_get_sourceimages(con, "MBO50#2025")
detections <- et_build_detections(parsed, si_ids)

## End(Not run)

Description

Aggregates a grouped data frame (from et_group_motion_images() or et_group_snapshot_images()) into one row per unique session_key, matching the data.sessions database schema.

Usage

et_build_sessions(grouped, deploymentid, type = "motion")

Arguments

Value

Data frame with columns: session_key (character, retained for joining), date (Date), deploymentid (character), type (character), starttime (POSIXct), endtime (POSIXct). One row per unique session_key.

Examples

## Not run: 
grouped <- et_group_motion_images(idx)
sessions <- et_build_sessions(grouped, deploymentid = "IT1#2023", type = "motion")

## End(Not run)

Description

Transforms a filtered ERDA index (files only) into the data.sourceimages schema used by the AMI database. Constructs public URLs from the full dir/name path in the index and extracts timestamps from AMI or CamAlien filenames.

Usage

et_build_sourceimages(
  index,
  sharelink,
  http_base = "https://anon.erda.au.dk/share_redirect",
  deploymentid,
  type = "snapshot"
)

Arguments

Details

The index_ts column records the time this function is called (i.e. the build timestamp), not the time the index was originally created.

Value

Data frame with columns: index_ts (character, build timestamp), filename, deploymentid, url, timestamp, issnapshot, processed, queued.

Examples

## Not run: 
idx <- et_index_filter("erda_index.parquet", extensions = c("jpg", "jpeg"))
si <- et_build_sourceimages(
  idx,
  sharelink    = Sys.getenv("ERDA_SHARELINK"),
  http_base    = "https://anon.erda.au.dk/share_redirect",
  deploymentid = "IT1#2023",
  type         = "snapshot"
)

## End(Not run)

Description

Transforms the output of et_parse_edge_tracks() into the data.tracks database schema by mapping id to edge_id and adding sessionid, algorithm, and algorithmversion columns.

Usage

et_build_tracks(
  parsed_tracks,
  sessionid,
  algorithm = "tracy",
  algorithmversion = "luke"
)

Arguments

Details

algorithm and algorithmversion are part of the data.tracks uniqueness key ⁠(sessionid, edge_id, algorithm, algorithmversion)⁠, which is ⁠NULLS DISTINCT⁠ (migration 2048). A NULL/empty identity would defeat the upsert de-dupe and insert duplicate tracks on re-ingest, so both are required to be non-empty.

The tracker identity is named ⁠tracy luke⁠: tracy is the tracker, and the version luke marks the on-edge tracker that predates kylo.

Value

A data frame with columns matching the data.tracks schema: sessionid, edge_id, algorithm, algorithmversion.

Examples

## Not run: 
parsed <- et_parse_edge_tracks("results/tracks/20250508TR.csv")
tracks <- et_build_tracks(parsed, sessionid = "abc-123")

## End(Not run)

Description

Wrapper around DBI::dbConnect() using RPostgres::Postgres(), parameterised by the ⁠CAMALIEN_*⁠ environment variables. This is intentionally separate from et_db_connect() (which targets the AMI database) so CamAlien writers can never accidentally connect to the wrong database — always call this function explicitly in CamAlien code paths (including inside mirai job bodies).

Usage

et_camalien_db_connect(
  host = Sys.getenv("CAMALIEN_HOST"),
  port = 5432L,
  dbname = Sys.getenv("CAMALIEN_DB", unset = "camalien"),
  user = Sys.getenv("CAMALIEN_USER"),
  password = Sys.getenv("CAMALIEN_PASSWORD"),
  sslmode = "verify-full",
  sslrootcert = Sys.getenv("CAMALIEN_SSLROOTCERT")
)

Arguments

Details

Requires the DBI and RPostgres packages at runtime.

Value

A DBI connection object.

Examples

## Not run: 
con <- et_camalien_db_connect()
on.exit(DBI::dbDisconnect(con))

## End(Not run)

Description

Given built CamAlien image rows, returns only those whose filename is not already in data2.imagefile. data2.imagefile.filename is globally unique (uq_v2_imagefile_filename), so it is the idempotency key — no session or partner scoping is required.

Usage

et_camalien_db_diff_images(dbcon, df, batch_size = 5000L)

Arguments

Value

The subset of df whose filenames are not yet in the database.

Examples

## Not run: 
new <- et_camalien_db_diff_images(con, imgs)

## End(Not run)

Description

Looks up a data.partner.id (UUID) in the camalien database, scoped to a project via projectSlug. camalien-db moved partner uniqueness to ⁠(storagePath, projectSlug)⁠, so the lookup is always project-scoped. This is a pure lookup — it never creates partners and errors if none matches.

Usage

et_camalien_db_get_partner(
  dbcon,
  partner,
  project_slug = "onestop",
  by = c("name", "partnerkey", "storagepath")
)

Arguments

Value

Character partner UUID.

Examples

## Not run: 
con <- et_camalien_db_connect()
pid <- et_camalien_db_get_partner(con, "Aarhus University")

## End(Not run)

Description

Resolves a data2.session.id (UUID) by ⁠(partnerid, name, year)⁠. This is a pure lookup — sessions are created by the backend, so it errors if no session matches, and also errors if more than one matches (there is no unique constraint on the triple; pass a session UUID directly in that case).

Usage

et_camalien_db_get_session(dbcon, partnerid, name, year)

Arguments

Value

Character session UUID.

Examples

## Not run: 
sid <- et_camalien_db_get_session(con, pid, "denmark", 2025L)

## End(Not run)

Description

Writes new CamAlien on-board image records into the data2 model in a single transaction, in FK-safe order: data2.imagedata (with a client-generated UUID id — the column has no DB default — and a PostGIS point built from lon/lat), then data2.additionalImagedata, then data2.imagefile. The data2.capturepoint row is created automatically by the trg_insert_capturepoint trigger and is never written here.

Usage

et_camalien_db_write_images(dbcon, df, batch_size = 250L)

Arguments

Details

data2.imagefile uses ⁠ON CONFLICT (filename) DO NOTHING⁠ as a safety net; call et_camalien_db_diff_images() first so only genuinely new images are written. When lon/lat are NA the point is inserted as SQL NULL.

Duplicate filenames within df are an error: data2.imagefile.filename is globally unique, so the second copy's imagefile insert would be silently dropped by ⁠ON CONFLICT DO NOTHING⁠, leaving its data2.imagedata row orphaned. et_build_camalien_images() deduplicates the index up front.

Requires the DBI and uuid packages at runtime.

Value

The number of images written (invisibly).

Examples

## Not run: 
con <- et_camalien_db_connect()
on.exit(DBI::dbDisconnect(con))
et_camalien_db_write_images(con, et_camalien_db_diff_images(con, imgs))

## End(Not run)

Description

After filtering stray images with et_filter_stray_sourceimages() or et_filter_stray_images(), this function examines the removed images to determine which years they belong to and whether matching deployments exist in the database. Emits informative cli messages and returns a summary data frame.

Usage

et_check_stray_deployments(removed, deployment_id, dbcon)

Arguments

Details

The lookup uses the projectid, partnerid, and code from the current deployment, substituting each stray year.

Value

Data frame with columns:

year

Integer. Year extracted from stray image timestamps.

n_images

Integer. Number of stray images in that year.

deployment_id

Character. The deployment ID if it exists, or NA_character_.

deployment_exists

Logical. Whether a deployment was found.

Returns a zero-row data frame if removed is empty.

Examples

## Not run: 
stray <- et_filter_stray_sourceimages(si, year = 2023)
if (nrow(stray$removed) > 0L) {
  info <- et_check_stray_deployments(stray$removed, "IT1#2023", con)
}

## End(Not run)

Description

Batch UPDATE that assigns sessionid to sourceimages based on temporal overlap with session ⁠(starttime, endtime)⁠ ranges. Motion sourceimages are matched to motion sessions, and snapshot sourceimages to snapshot sessions. Only sourceimages with ⁠sessionid IS NULL⁠ are updated, making this function idempotent.

Usage

et_db_assign_sessions(dbcon, deploymentid)

Arguments

Value

The number of rows updated (invisibly).

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
et_db_assign_sessions(con, "IT1#2023")

## End(Not run)

Description

Wrapper around DBI::dbConnect() using RPostgres::Postgres(). Connection parameters default to the environment variables used by the existing ami-indexer container (AMI_HOST, AMI_USER, AMI_PASSWORD, AMI_DB).

Usage

et_db_connect(
  host = Sys.getenv("AMI_HOST"),
  port = 5432L,
  dbname = Sys.getenv("AMI_DB", unset = "ami_ias"),
  user = Sys.getenv("AMI_USER"),
  password = Sys.getenv("AMI_PASSWORD"),
  sslmode = "verify-full",
  sslrootcert = Sys.getenv("AMI_SSLROOTCERT")
)

Arguments

Details

Requires the DBI and RPostgres packages at runtime.

Value

A DBI connection object.

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))

## End(Not run)

Description

Looks up a deployment's metadata and its associated project and partner folders to construct the canonical ERDA directory path. The path structure depends on the deployment's trap type:

Usage

et_db_deployment_path(dbcon, deployment_id)

Arguments

Details

• standalone/lepisense/mothbox: /project_folder/year/partner_folder/code/

• iot: /project_folder/code/ (no year or partner level; trap codes are unique within a project and images are organised in date-based subdirectories)

For standalone traps, a partner folder of "root" (or an unknown partner) is a sentinel meaning the deployment sits directly under the year, so the partner level is omitted (e.g. ⁠/vibes/2026/VIB01/⁠).

Value

Character string with the remote path, e.g. "/ias/2023/italy/IT1/" for standalone or "/ias/BIO25/" for IoT.

Examples

## Not run: 
con <- et_db_connect()
path <- et_db_deployment_path(con, "IT1#2023")
# "/ias/2023/italy/IT1/"

## End(Not run)

Description

Queries existing ⁠(code, year, projectid)⁠ triples from data.deployments and returns only the rows in df that do not yet exist.

Usage

et_db_diff_deployments(dbcon, df)

Arguments

Value

A subset of df containing only new deployments. All columns are preserved.

Examples

## Not run: 
new <- et_db_diff_deployments(con, df)
et_db_write_deployments(con, new)

## End(Not run)

Description

Compares a sourceimages data frame against data.sourceimages in the database and returns only the rows whose filename is not yet present. Queries are batched to stay within PostgreSQL's parameter limit.

Usage

et_db_diff_sourceimages(dbcon, df, batch_size = 5000L, n_workers = 1L)

Arguments

Value

A subset of df containing only rows not already in the database. All original columns are preserved.

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
si  <- et_build_sourceimages(idx, sharelink, http_base = http_base, deploymentid = 1L)
new <- et_db_diff_sourceimages(con, si)
et_db_write_sourceimages(con, new)

## End(Not run)

Description

Queries data.deployments by primary key and returns the deployment's year, code, partner ID, and project ID.

Usage

et_db_get_deployment(dbcon, deployment_id)

Arguments

Value

A named list with elements year (integer), code (character), partnerid (character), projectid (character), and traptype (character, e.g. "standalone", "iot", "lepisense", or "mothbox"). Aborts with an error if the deployment is not found.

Examples

## Not run: 
con <- et_db_connect()
dep <- et_db_get_deployment(con, "IT1#2023")
dep$year   # 2023
dep$code   # "IT1"

## End(Not run)

Description

Queries data.deployments for a deployment matching the given project, partner, year, and code. The deployment ID is a character string (e.g. "IT1#2023").

Usage

et_db_get_deployment_id(dbcon, projectid, partnerid, year, code)

Arguments

Value

Character deployment ID (e.g. "IT1#2023"), or NA_character_ if not found.

Examples

## Not run: 
con <- et_db_connect()
dep_id <- et_db_get_deployment_id(con, "ias", "8", 2023L, "IT1")

## End(Not run)

Description

Queries data.partners for the folder associated with a partner ID.

Usage

et_db_get_partner_folder(dbcon, partnerid)

Arguments

Value

Character string with the partner folder, or NA_character_ if not found.

Examples

## Not run: 
con <- et_db_connect()
folder <- et_db_get_partner_folder(con, 1L)

## End(Not run)

Description

Queries data.projects for the folder associated with a project ID.

Usage

et_db_get_project_folder(dbcon, projectid)

Arguments

Value

Character string with the project folder, or NA_character_ if not found.

Examples

## Not run: 
con <- et_db_connect()
folder <- et_db_get_project_folder(con, "ias")

## End(Not run)

Description

Queries data.sessions for all sessions belonging to a deployment. Used to retrieve sessions that were created by the database trigger (migration 2034) after sourceimages were inserted.

Usage

et_db_get_sessions(dbcon, deploymentid)

Arguments

Value

Data frame with columns: id (character, UUID), date (Date), type (character), starttime (POSIXct), endtime (POSIXct). Zero rows if no sessions exist.

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
sessions <- et_db_get_sessions(con, "IT1#2023")

## End(Not run)

Description

Queries data.sourceimages for all images belonging to a deployment, returning the filename (basename) and UUID of each. Used to resolve edge detection filenames to source-image UUIDs when writing data.detections.

Usage

et_db_get_sourceimages(dbcon, deploymentid)

Arguments

Value

Data frame with columns: id (character, UUID) and filename (character, basename). Zero rows if no source images exist.

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
si_ids <- et_db_get_sourceimages(con, "MBO50#2025")

## End(Not run)

Description

Queries data.tracks for all tracks belonging to a session, returning the track UUID keyed by ⁠(edge_id, algorithm, algorithmversion)⁠. Used to resolve edge track IDs to track UUIDs when writing the data.detection_tracks junction (the writers et_db_write_tracks() return only a row count).

Usage

et_db_get_tracks(dbcon, sessionid)

Arguments

Value

Data frame with columns: id (character, UUID), edge_id (integer), algorithm (character), algorithmversion (character). Zero rows if no tracks exist.

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
track_ids <- et_db_get_tracks(con, "abc-123")

## End(Not run)

Description

Queries data.deployments for all deployments matching an optional project and/or year filter. Useful for batch operations such as running the deploy pipeline over every deployment in a given project-year.

Usage

et_db_list_deployments(dbcon, projectid = NULL, year = NULL, traptype = NULL)

Arguments

Value

Character vector of deployment IDs ordered by id. Returns character(0) when no deployments match.

Examples

## Not run: 
con <- et_db_connect()
ids <- et_db_list_deployments(con, projectid = "ias", year = 2025L)

## End(Not run)

Description

Performs batch INSERT into data.classifications. On conflict with uq_classifications_detectionid_taxonlevel_algorithm (⁠detectionid, taxonlevel, algorithm⁠) the row is updated (DO UPDATE SET label, labelid, score, abovethreshold), so re-ingest refreshes scores. embeddingpath is left NULL (the edge pipeline provides none).

Usage

et_db_write_classifications(dbcon, df, batch_size = 250L)

Arguments

Value

Total number of rows affected (invisibly).

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
cls <- et_build_classifications(parsed, det_ids)
et_db_write_classifications(con, cls)

## End(Not run)

Description

Performs batch INSERT into data.deployments. Each deployment receives a UUID generated by uuid_generate_v4() in PostgreSQL. Geographic coordinates are converted to a PostGIS point via ST_SetSRID(ST_MakePoint(lon, lat), 4326).

Usage

et_db_write_deployments(dbcon, df, batch_size = 250L)

Arguments

Details

Use et_db_diff_deployments() beforehand to avoid inserting duplicates.

Value

Total number of rows inserted (invisibly).

Examples

## Not run: 
df <- et_build_deployments("deployments.csv", "ias", partner_map = pmap)
new <- et_db_diff_deployments(con, df)
et_db_write_deployments(con, new)

## End(Not run)

Description

Performs batch INSERT into data.detection_tracks. On conflict with the primary key ⁠(detectionid, trackid)⁠ the row is skipped (DO NOTHING), making re-ingest idempotent. The algorithm/algorithmversion on each row must equal the referenced track's identity or the composite foreign key to data.tracks fails.

Usage

et_db_write_detection_tracks(dbcon, df, batch_size = 250L)

Arguments

Value

Total number of rows affected (invisibly).

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
links <- et_build_detection_tracks(trs, det_ids, track_ids)
et_db_write_detection_tracks(con, links)

## End(Not run)

Description

Performs batch INSERT into data.detections. Each detection receives a UUID generated by uuid_generate_v4() in PostgreSQL (the id column has no default). On conflict with uq_detections_sourceimage_bbox (⁠sourceimageid, bbox_x1, bbox_y1, bbox_x2, bbox_y2⁠) the row is skipped (DO NOTHING), making re-ingest idempotent. After writing, the detection UUIDs are read back and matched onto the input rows by ⁠(sourceimageid, bbox_x1, bbox_y1, bbox_x2, bbox_y2)⁠, mirroring et_db_write_sessions().

Usage

et_db_write_detections(dbcon, df, batch_size = 250L)

Arguments

Details

contour, cropurl, and contour_geom are left NULL (the edge pipeline provides no polygon); area_px is a generated column and is never written.

Value

The input data frame with an id column (character UUID) populated from the database. Carries an n_written attribute: the number of rows actually inserted (rows skipped by the DO NOTHING conflict clause are not counted).

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
det <- et_build_detections(parsed, si_ids)
det_ids <- et_db_write_detections(con, det)

## End(Not run)

Description

Performs batch INSERT into data.sessions with ON CONFLICT upsert on ⁠(date, deploymentid, type)⁠. Returns the input data frame augmented with id (UUID) from the database.

Usage

et_db_write_sessions(dbcon, df, batch_size = 250L, n_workers = 1L)

Arguments

Value

The input data frame with an id column (character UUID) populated from the database.

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
sessions <- et_build_sessions(grouped, "IT1#2023", type = "motion")
sessions_with_ids <- et_db_write_sessions(con, sessions)

## End(Not run)

Description

Performs batch INSERT into data.sourceimages with ON CONFLICT upsert on ⁠(filename, deploymentid)⁠. Existing rows are updated with the new values. Uses parameterized queries via DBI::dbExecute().

Usage

et_db_write_sourceimages(dbcon, df, batch_size = 250L, n_workers = 1L)

Arguments

Value

The total number of rows affected (invisibly).

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
et_db_write_sourceimages(con, sourceimages_df)
et_db_write_sourceimages(con, sourceimages_df, n_workers = 4L)

## End(Not run)

Description

Performs batch upsert into data.tracking_tasks, the session-level tracking work queue. Used by the IoT path to record that a session's tracks were produced on the edge device: each row is written with status = "done" and the tracker algorithm/algorithmversion, which mark the edge provenance. The unique key is ⁠(sessionid, algorithm, algorithmversion)⁠ (constraint uq_tracking_tasks_session_algorithm), so on conflict the existing row's status is updated (DO UPDATE), making re-ingest idempotent. Every row needs a workerid — the tracking_tasks CHECK requires a non-NULL workerid for any non-pending status. Edge-tracked sessions have no server-side worker, so callers pass a fixed virtual edge-worker id (see .et_edge_workerid in erda_edge.R); workerid has no foreign key.

Usage

et_db_write_tracking_tasks(dbcon, df, batch_size = 250L)

Arguments

Value

Total number of rows affected (invisibly).

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
tasks <- data.frame(
  sessionid = "abc-123",
  status = "done",
  algorithm = "tracy",
  algorithmversion = "luke",
  workerid = "f2491a34-c18a-5188-984b-09092a9aaa55"
)
et_db_write_tracking_tasks(con, tasks)

## End(Not run)

Description

Performs batch INSERT into data.tracks. On conflict with the uq_tracks_session_edge_algorithm constraint (⁠sessionid, edge_id, algorithm, algorithmversion⁠) the row is skipped (DO NOTHING), making re-ingest idempotent. Typically used to write edge-processed tracking data from IoT AMI traps.

Usage

et_db_write_tracks(dbcon, df, batch_size = 250L)

Arguments

Value

Total number of rows affected (invisibly).

Examples

## Not run: 
con <- et_db_connect()
on.exit(DBI::dbDisconnect(con))
parsed <- et_parse_edge_tracks("results/tracks/20250508TR.csv")
tracks <- et_build_tracks(parsed, sessionid = "abc-123")
et_db_write_tracks(con, tracks)

## End(Not run)

Description

Returns the vector of ExifTool tag names passed to exifr::read_exif(). When the environment variable EXIF_TAGS is set (comma-separated tag names), it is used instead of the built-in defaults. "FileName" is always prepended so output parquet files retain their original name.

Usage

et_exif_tags()

Details

If EXIF_TAGS is set but produces no valid tag names after splitting and trimming (e.g. EXIF_TAGS=" , , "), a warning is emitted and the built-in defaults are used.

Default tags (when EXIF_TAGS is unset): "FileName", "Model", "CameraSerialNumber", "XResolution", "YResolution", "ImageWidth", "ImageHeight", "UserComment", "Duration", "VideoFrameRate".

Value

Character vector of tag names, always starting with "FileName".

Description

Keeps filenames ending in .jpg, .jpeg, .tif, .tiff, .mp4, .mov, or .avi (all case-insensitive).

Usage

et_filter_media_files(filenames)

Arguments

Value

Subset of filenames matching a supported extension.

Description

Removes rows from a grouped image data frame where the timestamp year does not match the deployment year exactly.

Usage

et_filter_stray_images(grouped, year, log_path = NULL)

Arguments

Details

Optionally writes the ERDA paths of removed images to a log file.

Value

A named list:

kept

Data frame of rows whose timestamps are within range.

removed

Data frame of rows that were filtered out.

Examples

## Not run: 
grouped <- et_group_motion_images(idx)
result <- et_filter_stray_images(grouped, year = 2025)
grouped <- result$kept

## End(Not run)

Description

Removes rows from a sourceimages data frame where the timestamp year does not match the deployment year exactly. This is the sourceimages-schema equivalent of et_filter_stray_images(), operating on character timestamps and the filename column instead of grouped data frames.

Usage

et_filter_stray_sourceimages(sourceimages, year, log_path = NULL)

Arguments

Value

A named list:

kept

Data frame of rows whose timestamps are within range.

removed

Data frame of rows that were filtered out.

Examples

## Not run: 
si <- et_build_sourceimages(idx, sharelink = "TOKEN",
  deploymentid = "IT1#2023", type = NULL)
result <- et_filter_stray_sourceimages(si, year = 2023)
si <- result$kept

## End(Not run)

Description

Takes an ERDA index data frame (as returned by et_index_filter()) and groups motion images into sessions using noon-to-noon grouping.

Usage

et_group_motion_images(index)

Arguments

Details

A session spans from noon UTC on one day to noon UTC on the next. The session_key column is the calendar date of the noon boundary, so an image at 22:00 on Aug 31 and another at 06:00 on Sep 1 both receive session_key = "2023_08_31". An image at 13:00 on Sep 1 starts a new session with key "2023_09_01".

Only motion images (as identified by et_parse_ami_filename()) are retained.

Value

The filtered data frame with three added columns:

session_key

Character. Session date as "YYYY_MM_DD".

timestamp

POSIXct (UTC). Parsed from the AMI filename.

trigger

Character. Trigger counter from the filename.

Examples

## Not run: 
idx <- et_index_filter("motion_index.parquet", extensions = c("jpg", "jpeg"))
grouped <- et_group_motion_images(idx)
table(grouped$session_key)

## End(Not run)

Description

Takes an ERDA index data frame (as returned by et_index_filter()) and groups snapshot images into sessions using noon-to-noon grouping.

Usage

et_group_snapshot_images(index)

Arguments

Details

A session spans from noon UTC on one day to noon UTC on the next. The session_key column is the calendar date of the noon boundary, so an image at 22:00 on Aug 31 and another at 06:00 on Sep 1 both receive session_key = "2023_08_31". An image at 13:00 on Sep 1 starts a new session with key "2023_09_01".

Only snapshot images (as identified by et_parse_ami_filename()) are retained.

Value

The filtered data frame with two added columns:

session_key

Character. Session date as "YYYY_MM_DD".

timestamp

POSIXct (UTC). Parsed from the AMI filename.

Examples

## Not run: 
idx <- et_index_filter("snapshot_index.parquet", extensions = c("jpg", "jpeg"))
grouped <- et_group_snapshot_images(idx)
table(grouped$session_key)

## End(Not run)

Description

Builds the HTTPS URL used to access a single image file through ERDA's anonymous share-redirect endpoint. This is a pure string operation and requires no network connection. Slashes between path components are normalised automatically.

Usage

et_img_url(
  sharelink,
  remote_dir,
  country,
  folder,
  filename,
  http_base = "https://anon.erda.au.dk/share_redirect"
)

Arguments

Value

A length-1 character vector containing the full URL.

Examples

et_img_url(
  sharelink  = "TOKEN",
  remote_dir = "/storage/2025/",
  country    = "portugal",
  folder     = "2025_07_10-11_10",
  filename   = "img.jpg"
)

Description

Traverses the directory tree rooted at remote_path using BFS, calling ls -la on each directory via et_sftp_batch(). Results are stored in a parquet file at index_path with columns dir, name, is_dir, size (integer, file size in bytes; NA for directories), and mtime (character, last-modified timestamp from ls -la, e.g. "Jul 10 11:10" or "Jan 5 2024").

Usage

et_index_dir(
  conn,
  remote_path,
  index_path,
  checkpoint_every = 25L,
  n_workers = 1L
)

Arguments

Details

On a fresh run, remote_path must exist: a missing root aborts with an error instead of producing an empty index.

Resume: if index_path already exists, the index is loaded and any directory already present in unique(index$dir) is skipped. Previously indexed directories are used to reconstruct the BFS queue without additional SFTP calls, so resuming after an interrupted run is fast.

Checkpointing: the parquet is rewritten every checkpoint_every directories. At most checkpoint_every directories of work can be lost if the connection drops between checkpoints.

Value

The full index data frame (invisibly).

Examples

## Not run: 
conn <- et_sftp_connect(Sys.getenv("ERDA_SHARELINK"))
on.exit(conn$disconnect(), add = TRUE, after = FALSE)
idx <- et_index_dir(conn, "/storage/2025", "./erda_index.parquet")
idx <- et_index_dir(conn, "/storage/2025", "./erda_index.parquet", n_workers = 4L)

## End(Not run)

Description

Removes directory rows from the index and optionally filters by file extension. Returns a filtered data frame with the same schema as the input.

Usage

et_index_filter(index, extensions = NULL)

Arguments

Value

Data frame with the same columns as the input index (dir, name, is_dir, size, mtime), filtered to files only.

Examples

## Not run: 
# From a parquet file
filtered <- et_index_filter("erda_index.parquet", extensions = c("jpg", "jpeg"))

# From an in-memory data frame
idx <- et_index_dir(conn, "/storage/2025", "erda_index.parquet")
filtered <- et_index_filter(idx, extensions = "jpg")

# All files (no extension filter)
all_files <- et_index_filter("erda_index.parquet")

## End(Not run)

Description

Re-runs ls -la on every directory in the existing index, diffs the listings against the stored entries, and returns a named list describing the changes. Newly discovered subdirectories are recursively indexed via et_index_dir().

Usage

et_index_update(conn, remote_path, index_path, update = TRUE)

Arguments

Value

Named list ⁠list(added = <data.frame>, removed = <data.frame>)⁠ (invisibly). Each data frame has columns dir, name, is_dir, size (integer, bytes), mtime (character timestamp).

Examples

## Not run: 
conn <- et_sftp_connect(Sys.getenv("ERDA_SHARELINK"))
on.exit(conn$disconnect(), add = TRUE, after = FALSE)
delta <- et_index_update(conn, "/storage/2025", "./erda_index.parquet")

## End(Not run)

Description

Creates a job in the in-memory job store with status "queued".

Usage

et_job_create(type, params = list())

Arguments

Value

Character string: the job ID.

Examples

id <- et_job_create("index_full", list(remote_path = "/storage/2025"))
et_job_get(id)

Description

If the job has a background process handle, its status is synced from the process state before returning.

Usage

et_job_get(id)

Arguments

Value

Named list with job details, or NULL if not found.

Examples

id <- et_job_create("test", list())
et_job_get(id)

Description

Syncs status from background process handles before returning.

Usage

et_job_list()

Value

A data frame with columns: id, type, status, stage, created_at, updated_at, last_progress_at. stage and last_progress_at are NA for jobs that have not reported progress. Returns an empty data frame if no jobs exist.

Examples

et_job_list()

Description

Background workers cannot write to the in-memory job store, so they report progress by writing a small JSON file that et_job_get() reads. This returns the path for a given job ID, under a per-process directory in base::tempdir(). Pass it to the worker and call et_progress_start() there.

Usage

et_job_progress_file(id)

Arguments

Value

Character string: absolute path of the progress file. The file itself only exists once the worker has reported progress.

Examples

et_job_progress_file("index_full_20260101_120000_abc123")

Description

Stores a mirai::mirai() handle on the job and sets the status to "running". The handle is polled by et_job_get() and et_job_list() to derive the current status.

Usage

et_job_set_process(id, process)

Arguments

Value

The updated job list (invisibly).

Description

Update a job's status

Usage

et_job_update(id, status, result = NULL, error = NULL)

Arguments

Value

The updated job list (invisibly).

Examples

id <- et_job_create("test", list())
et_job_update(id, "running")
et_job_update(id, "completed", result = list(rows = 100))

Description

Given a character vector of folder names and a local parquet directory, returns a named logical vector indicating which folders already have a corresponding .parquet file. This is a pure function: it performs no network I/O.

Usage

et_parquet_folders_done(folders, parquet_dir)

Arguments

Value

Named logical vector (names equal to folders). TRUE means a .parquet file already exists for that folder.

Examples

tmp <- tempdir()
writeLines("", fs::path(tmp, "2025_07_10-11_10.parquet"))
et_parquet_folders_done(c("2025_07_10-11_10", "2025_07_11-08_00"), tmp)

Description

AMI traps produce five filename formats:

Usage

et_parse_ami_filename(filenames)

Arguments

Details

• Motion (standalone): ⁠<YYYYMMDDHHmmss>-<sequence>-<trigger>.ext⁠ (e.g. ⁠20230831124747-00-01.jpg⁠)

• Motion (IoT): ⁠<YYYYMMDDHHmmss>.ext⁠ (e.g. ⁠20250508104842.jpg⁠) — bare 14-digit timestamp, no sequence or trigger counter

• Snapshot (with camera id): ⁠<camera_id>-<YYYYMMDDHHmmss>-<type>.ext⁠ (e.g. 01-20230831225959-snapshot.jpg)

• Snapshot (no camera id): ⁠<YYYYMMDDHHmmss>-<type>.ext⁠ (e.g. 20250527110000-snapshot.jpg)

• Mothbox: ⁠<trap_name>_<YYYY>_<MM>_<DD>__<HH>_<MM>_<SS>_<HDR>.ext⁠ (e.g. wryNinox_2026_05_29__23_02_29_HDR0.jpg) — scheduled captures with underscore-delimited date/time separated by double underscore

This function extracts the trigger counter (or camera ID for snapshots), timestamp, and image type from each filename. All formats are detected automatically. Filenames that match none of the patterns produce NA values for all columns.

Value

A data frame with columns trigger (character: trigger counter for motion images, camera ID for snapshot images that include one, or NA), timestamp (character, ⁠YYYY-MM-DD HH:MM:SS⁠), type (character, typically "snapshot" or "motion"), and sequence (character, motion sequence number or NA for snapshots). One row per filename.

Examples

et_parse_ami_filename(c(
  "01-20230831225959-snapshot.jpg",
  "20250527110000-snapshot.jpg",
  "20230831124747-00-01.jpg",
  "20250508104842.jpg",
  "wryNinox_2026_05_29__23_02_29_HDR0.jpg"
))

Description

CamAlien filenames encode metadata as underscore-delimited key-value pairs (e.g. CT_1234_GT_5678_lon_12.34_lat_56.78_alt_120.5_gain_1.4.jpg). This function strips the file extension and splits on ⁠_⁠, treating odd-positioned tokens as keys and even-positioned tokens as values.

Usage

et_parse_camalien_filename(filenames)

Arguments

Details

If the number of tokens is odd, the last value is padded with NA_character_ so that key-value pairing still works.

Known numeric columns (alt, vel, expt, gain) are coerced to double. Coordinates (lon, lat) and timestamps (CT, GT) are kept as character to preserve their original string representation. All other columns are character.

Note: Values containing underscores will cause incorrect key-value pairing, since ⁠_⁠ is also the key-value and pair delimiter.

Value

A tibble::tibble() with one row per filename and one column per key found in the filenames. Columns not present in a given filename are filled with NA. Column types: alt, vel, expt, gain are double; all others are character.

Examples

et_parse_camalien_filename(c(
  "CT_1234_GT_5678_lon_12.34_lat_56.78_alt_120.5_vel_0.0_expt_0.01_gain_1.4.jpg",
  "CT_9999_GT_0000_lon_8.10_lat_55.40_alt_80.0_vel_2.3_expt_0.02_gain_2.0.jpg"
))

Description

Reads a detection CSV produced by IoT AMI trap edge processing (e.g. results/detect/YYYYMMDD.csv). Parses timestamps and coerces column types.

Usage

et_parse_edge_detections(path)

Arguments

Value

A tibble::tibble() with columns:

year

Integer. Capture year.

trap

Character. Trap identifier.

timestamp

POSIXct (UTC). Capture time.

detect_conf

Double. Object detection confidence.

detect_id

Integer. Detection ID within image.

x1, y1, x2, y2

Double. Bounding box coordinates.

filename

Character. Source image filename.

order_label

Character. Taxonomic order label.

order_id

Integer. Order ID.

order_conf

Double. Order classification confidence.

above_th

Logical. Whether confidence exceeds threshold.

key

Integer. Detection key (links to track details).

species_label

Character. Species label.

species_id

Integer. Species ID.

species_conf

Double. Species classification confidence.

Examples

## Not run: 
detections <- et_parse_edge_detections("results/detect/20250508.csv")

## End(Not run)

Description

Reads a track detail CSV produced by IoT AMI trap edge processing (e.g. results/tracks/YYYYMMDDTRS.csv). One row per detection that belongs to a track; supplies the key -> track edge_id map used to build the data.detection_tracks junction.

Usage

et_parse_edge_track_detections(path)

Arguments

Value

A tibble::tibble() with columns:

track_edge_id

Integer. Track ID from the edge processor (joins to data.tracks.edge_id).

key

Integer. Detection key (joins to the detection CSV key).

timestamp

POSIXct (UTC). Detection time.

confidence

Double. Species classification confidence (percent).

valid

Logical. Whether confidence exceeds the threshold.

order

Character. Taxonomic order label.

species

Character. Species-level label.

xc, yc

Integer. Bounding-box centre (pixels).

x1, y1

Integer. Top-left corner of the bounding box (pixels).

width, height

Integer. Bounding-box dimensions (pixels).

x2, y2

Integer. Bottom-right corner (x1 + width, y1 + height).

image

Character. Source image filename (already a basename).

Examples

## Not run: 
trs <- et_parse_edge_track_detections("results/tracks/20250508TRS.csv")

## End(Not run)

Description

Reads a track summary CSV produced by IoT AMI trap edge processing (e.g. results/tracks/YYYYMMDDTR.csv). Parses timestamps and coerces column types.

Usage

et_parse_edge_tracks(path)

Arguments

Value

A tibble::tibble() with columns:

id

Integer. Track ID from edge processor.

starttime

POSIXct (UTC). Start of track.

endtime

POSIXct (UTC). End of track.

duration

Integer. Duration in seconds.

class

Character. Species label (majority vote).

counts

Integer. Number of frames in track.

percentage

Double. Classification consistency in percent.

size

Double. Mean bounding box area in pixels squared.

distance

Double. Total pixel displacement.

Examples

## Not run: 
tracks <- et_parse_edge_tracks("results/tracks/20250508TR.csv")

## End(Not run)

Description

Finds detection CSVs (results/detect/YYYYMMDD.csv) and track-detail CSVs (results/tracks/YYYYMMDDTRS.csv) in an ERDA index, downloads each via SFTP, and writes data.detections, data.classifications, and the data.detection_tracks junction. Source images, sessions, and tracks must already exist in the database (written upstream); this function fetches their UUIDs via et_db_get_sourceimages() and et_db_get_tracks().

Usage

et_process_edge_detections(
  conn,
  dbcon,
  idx_raw,
  sessions,
  deploymentid,
  algorithm = "tracy",
  algorithmversion = "luke",
  order_algorithm = "mcc24",
  species_algorithm = "uk-denmark-moths"
)

Arguments

Details

Writes happen in foreign-key-safe order: detections, then classifications, then the detection-track junction. Each detection CSV is matched to its track-detail CSV and motion session by the session date encoded in the filename.

Value

A named list with integer counts: detections, classifications, and detection_tracks (rows written).

Description

Finds track summary CSVs in an ERDA index, downloads each via SFTP, parses them with et_parse_edge_tracks(), maps tracks to motion sessions using the noon-to-noon date boundary, and writes to the database via et_db_write_tracks().

Usage

et_process_edge_tracks(
  conn,
  dbcon,
  idx_raw,
  sessions,
  algorithm = "tracy",
  algorithmversion = "luke"
)

Arguments

Value

Integer. Total number of track rows written.

Description

et_progress_start() points the current process at a job's progress file (from et_job_progress_file()); call it at the top of a mirai::mirai() job body. et_progress() then records the job's current stage and counters in that file, where et_job_get() picks them up and ⁠/index/status/{id}⁠ reports them as stage, progress and last_progress_at. et_progress_stop() detaches the reporter again.

Usage

et_progress_start(path)

et_progress_stop()

et_progress(stage, ..., interval = 2)

et_progress_read(path)

Arguments

Details

Long-running functions (et_index_dir(), et_index_update(), the edge processors and the database writers) call et_progress() themselves, so a job only needs to mark the stages between them. Outside a reporting job (no et_progress_start()), all calls are no-ops.

Writes are throttled to one every interval seconds unless the stage changes, and each write is atomic (temp file + rename), so a reader never sees a partial file.

Value

et_progress_start(), et_progress_stop() and et_progress(): NULL, invisibly. et_progress_read(): a list with stage, progress (named list of counters, or NULL), last_progress_at (POSIXct) and pid (the worker's process ID), or NULL if the file does not exist or cannot be parsed.

Examples

f <- tempfile(fileext = ".json")
et_progress_start(f)
et_progress("index", dirs_processed = 10L, files = 250L)
et_progress_read(f)
et_progress_stop()

Description

Executes one SFTP batch session over the existing ControlMaster connection returned by et_sftp_connect(), returning standard output as a character vector.

Usage

et_sftp_batch(conn, commands, stdout = TRUE, stderr = FALSE, timeout = 0)

Arguments

Value

Character vector of stdout lines (when stdout = TRUE), or the integer exit code (when stdout = FALSE). With stdout = TRUE, a non-zero exit status is attached as the "status" attribute.

Examples

## Not run: 
conn <- et_sftp_connect(Sys.getenv("ERDA_SHARELINK"))
on.exit(conn$disconnect(), add = TRUE, after = FALSE)
lines <- et_sftp_batch(conn, paste0("ls -1 /storage/2025/"))

## End(Not run)

Description

Establishes a persistent SSH connection to ERDA using an SSH ControlMaster socket. Authentication uses SSH_ASKPASS — no sshpass is required. The function waits up to 5 seconds for the socket to appear and calls cli::cli_abort() on timeout.

Usage

et_sftp_connect(
  sharelink,
  host = "io.erda.au.dk",
  port = "2222",
  tmp_root = fs::path(tempdir(), "sftp_imgs")
)

Arguments

Details

The returned connection object contains a ⁠$disconnect()⁠ closure that kills any tracked child SFTP processes, closes the ControlMaster, and removes the temporary directory. Register it in the calling frame with:

conn <- et_sftp_connect(sharelink, host, port)
on.exit(conn$disconnect(), add = TRUE, after = FALSE)

Value

A named list (connection object) with elements:

ctl_opts

Character vector of SSH -o flags for multiplexing.

remote_target

String "<sharelink>@<host>".

tmp_root

Path to the temporary directory.

port

The SFTP port string.

add_child_pid

⁠function(pid)⁠ — register a background SFTP PID for cleanup on disconnect.

remove_child_pids

⁠function(pids)⁠ — deregister finished PIDs.

disconnect

Parameterless function — kill tracked children, close the ControlMaster, and remove tmp_root.

Examples

## Not run: 
conn <- et_sftp_connect(
  sharelink = Sys.getenv("ERDA_SHARELINK"),
  host      = Sys.getenv("ERDA_SFTP_HOST", unset = "io.erda.au.dk"),
  port      = Sys.getenv("ERDA_SFTP_PORT", unset = "2222")
)
on.exit(conn$disconnect(), add = TRUE, after = FALSE)

## End(Not run)

Description

Calls et_sftp_list_folders() and filters results to entries whose base::basename() matches the YYYY_MM_DD-HH_MM pattern used by CamAlien onboard systems.

Usage

et_sftp_list_dated_subfolders(conn, country_path)

Arguments

Value

Character vector of matching folder names (or full paths, depending on what the SFTP server returns).

Examples

## Not run: 
conn <- et_sftp_connect(Sys.getenv("ERDA_SHARELINK"))
on.exit(conn$disconnect(), add = TRUE, after = FALSE)
subfolders <- et_sftp_list_dated_subfolders(conn, "/storage/2025/portugal")

## End(Not run)

Description

Runs ⁠ls -1 <remote_path>⁠ over the SFTP connection and returns the non-blank, non-prompt lines.

Usage

et_sftp_list_folders(conn, remote_path)

Arguments

Value

Character vector of folder/file names returned by the server.

Examples

## Not run: 
conn <- et_sftp_connect(Sys.getenv("ERDA_SHARELINK"))
on.exit(conn$disconnect(), add = TRUE, after = FALSE)
folders <- et_sftp_list_folders(conn, "/storage/2025/portugal")

## End(Not run)

Description

Each element of user_comment is expected to be a JSON string of the form:

{"gain": "1.4", "detections": ["1359658 Morus alba L. 9.97668 8.829123", ...]}

Each detection string has the layout: ⁠<plantnet_id> <species tokens...> <probability> <score>⁠

Usage

parse_detections(user_comment)

Arguments

Details

The function is designed to be used with tidyr::unnest() on a list column. NA, empty, or malformed entries yield a zero-row data frame.

Value

A list of tibble::tibble()s, one per element of user_comment, each with columns plantnet_id (integer), species (character), probability (double), score (double).

Description

Strips ⁠sftp>⁠ prompt lines, blank lines, and ./.. entries from the raw character vector returned by et_sftp_batch() when running ⁠ls -la <path>⁠. Parses the permission string to determine entry type.

Usage

parse_sftp_ls_la(raw_lines)

Arguments

Details

Each line must have at least 9 whitespace-separated fields: ⁠<perms> <links> <owner> <group> <size> <month> <day> <time> <name...>⁠. Lines with fewer fields are silently dropped.

For symlinks (lrwxrwxrwx), the ⁠ -> target⁠ suffix is stripped from the name and the entry is treated as a regular file (is_dir = FALSE).

Value

Data frame with columns name (character), is_dir (logical), size (integer – file size in bytes, NA for directories), mtime (character – last-modified timestamp from ls -la, fields 6-8 joined with a space, e.g. "Jul 10 11:10" or "Jan 5 2024").