Skip to main content

Troubleshooting and Incident Resolution Guide

This guide compiles field-tested solutions for diagnosing and resolving cross-cutting anomalies across the TkushPatch platform.

1. CLI and Compilation Anomalies​

Error: shorebird.yaml not found or Missing app_id​

  • Symptom: Running tk-ota-push release or tk-ota-push patch aborts with an error indicating a missing configuration manifest.
  • Root Cause: The command is executing from a nested subfolder or the project was never initialized.
  • Resolution:
    1. Navigate to the root directory containing pubspec.yaml.
    2. Run tk-ota-push init to regenerate shorebird.yaml.
    3. Ensure both mandatory properties are present: 
      app_id: "YOUR-APP-UUID"
      base_url: "https://api.tkushpatch.domain.com"

Error: Patch rejected: Native code modified (native_bridge_ok: false)​

  • Symptom: The tk-ota-push patch command halts, flagging alterations to native platform code.
  • Root Cause: Dependencies added to pubspec.yaml introduce C++, Java/Kotlin, or Objective-C/Swift native source files absent from the Store release.
  • Resolution:
    • An OTA patch can only deliver pure Dart code.
    • If you introduced native plugins (e.g., a hardware camera or BLE library), you must compile a new baseline Store release (tk-ota-push release android) and submit it to marketplace review.

2. Distribution and Patch Lifecycle Issues​

Symptom: The patch is published but devices do not download it​

Work systematically through these 5 common checks:

Troubleshooting Decision Flow
  • Patch status is STAGED? ➔ Go to Approvals in Web Console to review and approve the patch.
  • Patch status is ROLLED_BACK? ➔ Update was revoked. Publish a new incremental patch number.
  • Device does not receive update? ➔ Check device channel, targeted countries, and active rollout percentage.

HTTP 429 Too Many Requests​

  • Symptom: Handsets receive an HTTP 429 status code with a Retry-After: 60 response header.
  • Root Cause: The Token Bucket Rate-Limiter (DataPlaneRateLimitFilter) detected more than 120 requests/minute from the client IP address.
  • Resolution:
    • If mobile devices share a centralized corporate egress proxy, increase the capacity threshold and trust proxy headers in application.properties: 
      ota.rate-limit.trust-forwarded-header=true
      ota.rate-limit.capacity-per-minute=300

3. Cryptography and cryptographic signing engine Signer​

Error: RSA signature verification failed (ring::error::Unspecified)​

  • Symptom: The mobile handset downloads the patch but rejects it, logging a verification failure in adb logcat or Xcode Console.
  • Root Cause 1: The public key baked into the initial Store release binary does not match the private key used to sign the patch.
    • Verification: Did you build the APK with a different account or point to a non-production environment key?
  • Root Cause 2: Signature payload mismatch. The server must sign the 64-character lowercase ASCII representation of the SHA-256 digest (hash.toLowerCase().getBytes(US_ASCII)), not the raw binary bytes.

Console Error: Signing Service Unavailable (HTTP 503)​

  • Symptom: Approving or signing patches on /approvals fails with a 503 Service Unavailable alert.
  • Root Cause: The cryptographic signing engine container restarted and returned to the initial sealed state.
  • Resolution:
    1. Access the signing engine cluster via SSH or admin terminal.
    2. Submit your Shamir unseal quorum: Contact your system administrator to reactivate the signing service.
    3. Once sealed: false, reload the web console to resume signing operations.

4. Mobile Runtime Behavior and Crash Recovery​

Symptom: The patch downloads successfully but changes do not appear​

  • Root Cause: By default, the Flutter runtime activates downloaded patches upon the next cold restart of the application to prevent jarring runtime state loss.
  • Resolution:
    1. Terminate the app completely (swipe it away from the OS multitasking switcher).
    2. Relaunch it. The update should execute immediately.
    3. To enforce an immediate in-session restart, invoke TkOtaPush.restartApp() from the SDK within a custom blocking update dialog.

Symptom: Application crashes at startup and reverts to older code​

  • Root Cause: TkushPatch's native 2-Crash Rollback Guard caught an unhandled fatal exception on consecutive startup cycles.
  • Diagnostic Steps:
    1. Navigate to /symbols.
    2. Capture the crash stack trace from device logs or your APM reporter.
    3. Paste the obfuscated stack trace into the deobfuscation tool to pinpoint the offending source line: 
      #00 abs 00000072a3194 application binary + 0x12a3194
      => package:my_app/services/api_service.dart:42:15 in fetchData()
    4. Fix the Dart exception in code and publish a replacement Patch #2.