Improve video sending pipeline: preprocessing + typed upload progress

.agent-docs/2026-02-22-awesome-video-send-non-ui-plan.md

@@ -0,0 +1,81 @@
+# Awesome Video Sending (Thread 1061) - Non-UI Plan
+
+## Scope and Goal
+- Scope: shared video send/upload pipeline only (InlineKit + API interactions), no picker redesign and no "Library merge" UX work.
+- Goal: reach thread targets for reliable long video send, pre-upload processing/compression, true transfer progress data, and processing/upload byte state availability for message surfaces.
+
+## Source Inputs Reviewed
+- Thread data: chat `1061` (`spec: awesome video sending`) via Inline CLI.
+- Telegram references (local):
+  - `/Users/dena/dev/Telegram-iOS/submodules/LegacyComponents/Sources/TGMediaVideoConverter.m`
+  - `/Users/dena/dev/Telegram-iOS/submodules/TelegramCore/Sources/State/PendingMessageManager.swift`
+  - `/Users/dena/dev/Telegram-iOS/submodules/TelegramUI/Components/Chat/ChatMessageInteractiveMediaNode/Sources/ChatMessageInteractiveMediaNode.swift`
+- Inline architecture:
+  - `apple/InlineKit/Sources/InlineKit/Files/FileCache.swift`
+  - `apple/InlineKit/Sources/InlineKit/Files/VideoCompression.swift`
+  - `apple/InlineKit/Sources/InlineKit/Files/FileUpload.swift`
+  - `apple/InlineKit/Sources/InlineKit/ApiClient.swift`
+  - `server/src/methods/uploadFile.ts`
+- Best-practice references:
+  - Apple AVFoundation export/session state + progress + network optimization docs.
+  - Apple URLSessionTaskDelegate upload progress callback docs (`didSendBodyData`).
+
+## Facts From Thread 1061 (Must-Haves)
+- Pre-upload video processing/compression for faster uploads.
+- Real upload progress (not fake spinner-only state).
+- Report uploaded bytes and total bytes in upload state.
+- Communicate processing state during resize/compression.
+- Keep stage 3 out of scope for this PR:
+  - multipart upload
+  - resumable upload after app restart
+  - major server/client large-file infrastructure rework
+- Pipeline is shared with macOS; solution must be shared-safe.
+- Add tests and ensure they pass.
+
+## Current Gaps
+- MP4s are currently attached as-is in `FileCache.saveVideo`, so most camera videos skip compression.
+- Upload progress is a `Double` fraction with legacy sentinel `-1` for processing.
+- No first-class bytes-sent / bytes-total model for upload.
+- Video upload state is not exposed as structured progress stream for consumers.
+
+## Non-UI Implementation Plan
+1. Add typed upload progress model in `InlineKit`.
+   - Introduce a single source-of-truth upload progress snapshot with stages: `processing`, `uploading`, `completed`, `failed`.
+   - Include `bytesSent`, `totalBytes`, and clamped fraction.
+
+2. Upgrade API upload delegate progress payload.
+   - Replace raw fraction callback with structured transport progress from `ApiClient.uploadFile`.
+   - Keep behavior compatible and deterministic (`0` at start, `1` on success).
+
+3. Move video preprocessing into upload pipeline.
+   - Keep `FileCache.saveVideo` for secure copy/transcode normalization and local preview persistence.
+   - In `FileUploader.performUpload`, preprocess video for upload before network transfer:
+     - attempt compression for large/high-bitrate videos
+     - fall back safely to original local file when compression is unnecessary/ineffective
+     - propagate processing state before upload starts
+     - use actual upload payload size for byte totals exposed to progress state
+
+4. Improve cancellation behavior during video export.
+   - Ensure compression/export is cancellation-aware, so canceling send aborts preprocessing promptly.
+
+5. Expose upload progress stream APIs from `FileUploader`.
+   - Add publisher API per media id (initially video path needed for this task).
+   - Maintain existing lifecycle semantics for completion/failure and cleanup.
+
+6. Add tests.
+   - Unit tests for upload progress stage/fraction/byte mapping behavior.
+   - Keep existing `VideoCompressionTests` passing.
+
+## Risk Controls
+- Do not modify server contract; send same multipart fields (`type`, `file`, video metadata + thumbnail).
+- Do not implement multipart/resume in this PR.
+- Keep shared pipeline behavior valid for both iOS and macOS.
+
+## Validation Plan
+- `cd apple/InlineKit && swift test --filter VideoCompressionTests`
+- `cd apple/InlineKit && swift test --filter FileUploadProgressTests`
+- If filters are unavailable in this environment, run full `swift test` for InlineKit.
+
+## Review Checkpoint (Deep Understanding)
+- This plan directly maps to the highest-priority thread asks (compression, processing, real upload progress with bytes) and intentionally excludes stage-3 architecture work.
+- The approach is incremental: no API break on server side, minimal behavior change surface outside upload pipeline, and test-backed progress semantics.

apple/InlineIOS/Features/Media/NewVideoView.swift

@@ -13,7 +13,11 @@ final class NewVideoView: UIView {
   private let cornerRadius: CGFloat = 16.0
   private let maskLayer = CAShapeLayer()
   private var imageConstraints: [NSLayoutConstraint] = []
-  private var progressCancellable: AnyCancellable?
+  private var downloadProgressCancellable: AnyCancellable?
+  private var uploadProgressCancellable: AnyCancellable?
+  private var uploadProgressBindingTask: Task<Void, Never>?
+  private var uploadProgressLocalId: Int64?
+  private var uploadProgressSnapshot: UploadProgressSnapshot?
   private var isDownloading = false
   private var isPresentingViewer = false
   private var pendingViewerURL: URL?
@@ -143,7 +147,9 @@ final class NewVideoView: UIView {
   }
 
   deinit {
-    progressCancellable?.cancel()
+    downloadProgressCancellable?.cancel()
+    uploadProgressCancellable?.cancel()
+    uploadProgressBindingTask?.cancel()
   }
 
   // MARK: - Setup
@@ -267,6 +273,7 @@ final class NewVideoView: UIView {
     setupMask()
     setupGestures()
     updateImage()
+    syncUploadProgressBinding()
     updateDurationLabel()
     updateOverlay()
   }
@@ -318,6 +325,7 @@ final class NewVideoView: UIView {
     let prev = self.fullMessage
     self.fullMessage = fullMessage
     updateMask()
+    syncUploadProgressBinding()
 
     if
       prev.videoInfo?.id == fullMessage.videoInfo?.id,
@@ -329,8 +337,8 @@ final class NewVideoView: UIView {
       return
     }
 
-    progressCancellable?.cancel()
-    progressCancellable = nil
+    downloadProgressCancellable?.cancel()
+    downloadProgressCancellable = nil
     isDownloading = false
     downloadProgress = 0
     downloadProgressView.setProgress(0)
@@ -351,7 +359,9 @@ final class NewVideoView: UIView {
 
   private func updateOverlay() {
     let isVideoDownloaded = hasLocalVideoFile()
-    let isUploading = fullMessage.message.status == .sending && fullMessage.videoInfo?.video.cdnUrl == nil
+    let isUploading = isPendingUpload()
+      || uploadProgressSnapshot?.stage == .processing
+      || uploadProgressSnapshot?.stage == .uploading
     let globalDownloadActive = fullMessage.videoInfo
       .map { FileDownloader.shared.isVideoDownloadActive(videoId: $0.id) } ?? false
     let downloading = !isVideoDownloaded && (isDownloading || globalDownloadActive)
@@ -385,6 +395,19 @@ final class NewVideoView: UIView {
   }
 
   private func updateDurationLabel() {
+    if isPendingUpload(), let uploadProgressSnapshot {
+      durationBadge.isHidden = false
+      switch uploadProgressSnapshot.stage {
+      case .processing:
+        durationBadge.text = "Processing"
+      case .uploading, .completed:
+        durationBadge.text = uploadProgressLabel(uploadProgressSnapshot)
+      case .failed:
+        durationBadge.text = "Failed"
+      }
+      return
+    }
+
     guard let duration = fullMessage.videoInfo?.video.duration, duration > 0 else {
       durationBadge.isHidden = true
       return
@@ -405,6 +428,27 @@ final class NewVideoView: UIView {
     return String(format: "%d:%02d", mins, secs)
   }
 
+  private func uploadProgressLabel(_ progress: UploadProgressSnapshot) -> String {
+    if progress.totalBytes > 0 {
+      return "\(formatTransferBytes(progress.bytesSent))/\(formatTransferBytes(progress.totalBytes))"
+    }
+
+    if progress.fractionCompleted > 0 {
+      let percent = Int((progress.fractionCompleted * 100).rounded())
+      return "\(percent)%"
+    }
+
+    return "Uploading"
+  }
+
+  private func formatTransferBytes(_ bytes: Int64) -> String {
+    ByteCountFormatter.string(fromByteCount: max(0, bytes), countStyle: .file)
+  }
+
+  private func isPendingUpload() -> Bool {
+    fullMessage.message.status == .sending && fullMessage.videoInfo?.video.cdnUrl == nil
+  }
+
   // MARK: - Playback
 
   @objc private func handleTap() {
@@ -470,15 +514,15 @@ final class NewVideoView: UIView {
   }
 
   private func bindProgressIfNeeded(videoId: Int64) {
-    guard progressCancellable == nil else { return }
+    guard downloadProgressCancellable == nil else { return }
 
-    progressCancellable = FileDownloader.shared.videoProgressPublisher(videoId: videoId)
+    downloadProgressCancellable = FileDownloader.shared.videoProgressPublisher(videoId: videoId)
       .receive(on: DispatchQueue.main)
       .sink { [weak self] progress in
         guard let self else { return }
 
         if let local = self.videoLocalUrl(), FileManager.default.fileExists(atPath: local.path) {
-          self.progressCancellable = nil
+          self.downloadProgressCancellable = nil
           self.isDownloading = false
           self.downloadProgress = 0
           self.updateOverlay()
@@ -491,7 +535,7 @@ final class NewVideoView: UIView {
         }
 
         if progress.error != nil || progress.isComplete {
-          self.progressCancellable = nil
+          self.downloadProgressCancellable = nil
           self.isDownloading = false
           self.downloadProgress = 0
           self.updateOverlay()
@@ -502,14 +546,54 @@ final class NewVideoView: UIView {
   @objc private func handleCancelDownload() {
     guard let videoId = fullMessage.videoInfo?.id else { return }
     FileDownloader.shared.cancelVideoDownload(videoId: videoId)
-    progressCancellable?.cancel()
-    progressCancellable = nil
+    downloadProgressCancellable?.cancel()
+    downloadProgressCancellable = nil
     isDownloading = false
     downloadProgress = 0
     downloadProgressView.setProgress(0)
     updateOverlay()
   }
 
+  private func syncUploadProgressBinding() {
+    guard isPendingUpload(), let videoLocalId = fullMessage.videoInfo?.video.id else {
+      clearUploadProgressBinding(resetState: true)
+      return
+    }
+
+    if uploadProgressLocalId == videoLocalId, (uploadProgressCancellable != nil || uploadProgressBindingTask != nil) {
+      return
+    }
+
+    clearUploadProgressBinding(resetState: false)
+    uploadProgressLocalId = videoLocalId
+    uploadProgressBindingTask = Task { @MainActor [weak self] in
+      guard let self else { return }
+      let publisher = await FileUploader.shared.videoProgressPublisher(videoLocalId: videoLocalId)
+      guard !Task.isCancelled, self.uploadProgressLocalId == videoLocalId else { return }
+
+      self.uploadProgressBindingTask = nil
+      self.uploadProgressCancellable = publisher
+        .receive(on: DispatchQueue.main)
+        .sink { [weak self] progress in
+          guard let self else { return }
+          self.uploadProgressSnapshot = progress
+          self.updateDurationLabel()
+          self.updateOverlay()
+        }
+    }
+  }
+
+  private func clearUploadProgressBinding(resetState: Bool) {
+    uploadProgressBindingTask?.cancel()
+    uploadProgressBindingTask = nil
+    uploadProgressCancellable?.cancel()
+    uploadProgressCancellable = nil
+    uploadProgressLocalId = nil
+    if resetState {
+      uploadProgressSnapshot = nil
+    }
+  }
+
   private func videoLocalUrl() -> URL? {
     if let localPath = fullMessage.videoInfo?.video.localPath {
       return FileCache.getUrl(for: .videos, localPath: localPath)

apple/InlineKit/Sources/InlineKit/ApiClient.swift

@@ -73,10 +73,23 @@ public final class ApiClient: ObservableObject, @unchecked Sendable {
   public init() {}
 
   private let log = Log.scoped("ApiClient")
+
+  public struct UploadTransferProgress: Sendable, Equatable {
+    public let bytesSent: Int64
+    public let totalBytes: Int64
+    public let fractionCompleted: Double
+
+    public init(bytesSent: Int64, totalBytes: Int64, fractionCompleted: Double) {
+      self.bytesSent = max(0, bytesSent)
+      self.totalBytes = max(0, totalBytes)
+      self.fractionCompleted = min(max(fractionCompleted, 0), 1)
+    }
+  }
+
   private final class UploadTaskDelegate: NSObject, URLSessionTaskDelegate {
-    private let progressHandler: @Sendable (Double) -> Void
+    private let progressHandler: @Sendable (UploadTransferProgress) -> Void
 
-    init(progressHandler: @escaping @Sendable (Double) -> Void) {
+    init(progressHandler: @escaping @Sendable (UploadTransferProgress) -> Void) {
       self.progressHandler = progressHandler
     }
 
@@ -87,9 +100,16 @@ public final class ApiClient: ObservableObject, @unchecked Sendable {
       totalBytesSent: Int64,
       totalBytesExpectedToSend: Int64
     ) {
-      guard totalBytesExpectedToSend > 0 else { return }
-      let fraction = Double(totalBytesSent) / Double(totalBytesExpectedToSend)
-      progressHandler(min(max(fraction, 0), 1))
+      let fraction = totalBytesExpectedToSend > 0
+        ? Double(totalBytesSent) / Double(totalBytesExpectedToSend)
+        : 0
+      progressHandler(
+        UploadTransferProgress(
+          bytesSent: totalBytesSent,
+          totalBytes: totalBytesExpectedToSend,
+          fractionCompleted: fraction
+        )
+      )
     }
   }
 
@@ -666,7 +686,7 @@ public final class ApiClient: ObservableObject, @unchecked Sendable {
     filename: String,
     mimeType: MIMEType,
     videoMetadata: VideoUploadMetadata? = nil,
-    progress: @escaping @Sendable (Double) -> Void
+    progress: @escaping @Sendable (UploadTransferProgress) -> Void
   ) async throws -> UploadFileResult {
     guard let url = URL(string: "\(baseURL)/uploadFile") else {
       throw APIError.invalidURL
@@ -729,7 +749,8 @@ public final class ApiClient: ObservableObject, @unchecked Sendable {
     do {
       let delegate = UploadTaskDelegate(progressHandler: progress)
       let session = URLSession(configuration: .default, delegate: delegate, delegateQueue: nil)
-      progress(0)
+      let totalBodyBytes = Int64(multipartFormData.body.count)
+      progress(UploadTransferProgress(bytesSent: 0, totalBytes: totalBodyBytes, fractionCompleted: 0))
       let (data, response) = try await session.upload(for: request, from: multipartFormData.body)
       session.finishTasksAndInvalidate()
 
@@ -742,7 +763,7 @@ public final class ApiClient: ObservableObject, @unchecked Sendable {
           let apiResponse = try decoder.decode(APIResponse<UploadFileResult>.self, from: data)
           switch apiResponse {
             case let .success(data):
-              progress(1)
+              progress(UploadTransferProgress(bytesSent: totalBodyBytes, totalBytes: totalBodyBytes, fractionCompleted: 1))
               return data
             case let .error(error, errorCode, description):
               log.error("Error \(error): \(description ?? "")")