BiospexZipMerger

Purpose

Consolidates multiple partial ZIP files (created by BiospexZipCreator) into a single final archive.

Workflow

1. Trigger: Manually invoked or triggered by Laravel when all partial ZIPs for an export are ready.
2. Download & Extract: Downloads each partial ZIP from S3 and extracts its contents in memory/temp space.
3. Re-compression: Appends all files into a new archiver stream.
4. Upload: Streams the final consolidated ZIP back to S3.
5. Cleanup: Deletes the source partial ZIP files from S3 to save space.
6. Callback: Sends a zip-ready status update to the Laravel app via SQS.

Inputs/Outputs

• Inputs (JSON):
  • bucket: The S3 bucket containing the ZIP parts.
  • partKeys: Array of objects containing zipKey for each part.
  • finalKey: The target S3 key for the merged archive.
  • updatesQueueUrl: SQS URL for reporting status.
  • queueId: Reference ID for the Laravel export queue.
• Outputs:
  • S3: The final merged ZIP file at finalKey.
  • SQS: Status notification to updatesQueueUrl.

Related Components

• Laravel Command: App\Console\Commands\SqsListenerExportUpdate (Processes the final zip-ready status).
• Related Lambda: BiospexZipCreator (Source of the partial ZIP files).

Deployment

Use the deploy.sh script for interactive deployment to AWS (Region: us-east-2).