Overview

Impact: High - Prevents testing iOS SDK features in simulator
Priority: P1
Effort: Medium

The iOS sample app fails to run properly in the iOS simulator while working correctly on physical devices. Specifically, the app crashes or system functionality fails due to missing or incorrectly bundled native library binaries for simulator architectures.

1. Problem Statement

What's the Issue?

The iOS SDK and sample app do not work properly when running in the iOS Simulator. System details and core SDK functionality fail, while the same code works perfectly on physical iPhone devices.

Why Does It Matter?

• Blocks development workflow: Developers cannot test features in the simulator, requiring physical devices for all testing
• Slows down iteration: Simulator testing is faster and more convenient than physical device testing
• CI/CD limitations: Automated testing in simulator environments will fail
• Poor developer experience: Forces developers to have physical devices available at all times

What's Broken?

The root cause appears to be that native library binaries (LlamaCPP and ONNX backends) are not being properly bundled for simulator architectures (x86_64 for Intel Macs, arm64 for Apple Silicon simulators). The XCFrameworks may be missing simulator slices or have incorrect architecture linkage.

2. Current State

Affected Components

• sdk/runanywhere-swift/ - Swift SDK package
• sdk/runanywhere-commons/ - Native C++ library with backends
• examples/ios/RunAnywhereAI/ - Sample iOS application

Build Script Locations

sdk/runanywhere-commons/scripts/build-ios.sh
sdk/runanywhere-swift/scripts/build-swift.sh

Current Build Process

The build script (build-ios.sh) does build for three platforms:

• OS → ios-arm64 (physical device)
• SIMULATORARM64 → ios-arm64_x86_64-simulator (Apple Silicon simulator)
• SIMULATOR → ios-arm64_x86_64-simulator (Intel simulator)

Lines 470-473 in sdk/runanywhere-commons/scripts/build-ios.sh:

build_platform "OS"
build_platform "SIMULATORARM64"
build_platform "SIMULATOR"

XCFramework Structure

Current XCFrameworks are created with both slices:

Binaries/
├── RACommons.xcframework/
│   ├── ios-arm64/ (device)
│   └── ios-arm64_x86_64-simulator/ (simulator)
├── RABackendLLAMACPP.xcframework/
│   ├── ios-arm64/
│   └── ios-arm64_x86_64-simulator/
└── RABackendONNX.xcframework/
    ├── ios-arm64/
    └── ios-arm64_x86_64-simulator/

The Problem

Despite having simulator slices, the XCFrameworks may contain:

1. Missing symbols: Some backend libraries might not be included in simulator builds
2. Architecture mismatch: Dependencies like Sherpa-ONNX or llama.cpp may not be properly linked for simulator architectures
3. Incomplete fat binary: The lipo command may be failing silently or not including all required libraries

Lines 324-340 in build-ios.sh show ONNX backend bundling:

elif [[ "$BACKEND_NAME" == "onnx" ]]; then
    local SHERPA_XCFW="${PROJECT_ROOT}/third_party/sherpa-onnx-ios/sherpa-onnx.xcframework"
    local SHERPA_ARCH
    case $PLATFORM in
        OS) SHERPA_ARCH="ios-arm64" ;;
        *) SHERPA_ARCH="ios-arm64_x86_64-simulator" ;;
    esac
    # Extract library from Sherpa-ONNX
    for possible in \
        "${SHERPA_XCFW}/${SHERPA_ARCH}/libsherpa-onnx.a" \
        "${SHERPA_XCFW}/${SHERPA_ARCH}/sherpa-onnx.framework/sherpa-onnx"; do
        if [[ -f "$possible" ]]; then
            LIBS_TO_BUNDLE+=("$possible")
            break
        fi
    done
fi

Sample App Issue

File: examples/ios/RunAnywhereAI/RunAnywhereAI/Core/Services/DeviceInfoService.swift

The app uses native system calls that may fail on simulator if underlying SDK is not properly initialized:

private func getDeviceModelName() async -> String {
    var systemInfo = utsname()
    uname(&systemInfo)
    // ... depends on SDK being properly loaded
}

3. Proposed Solution

Verify and Fix XCFramework Bundling

Step 1: Validate Third-Party Dependencies

Ensure Sherpa-ONNX and llama.cpp dependencies support simulator architectures:

# Check Sherpa-ONNX xcframework
cd sdk/runanywhere-commons/third_party/sherpa-onnx-ios
xcodebuild -create-xcframework -help
lipo -info sherpa-onnx.xcframework/ios-arm64_x86_64-simulator/sherpa-onnx.framework/sherpa-onnx

# Should show: x86_64 arm64

Step 2: Fix Backend Bundling Logic

Update sdk/runanywhere-commons/scripts/build-ios.sh to ensure all dependencies are included:

# Around line 344-350, add validation
if [[ ${#LIBS_TO_BUNDLE[@]} -gt 0 ]]; then
    log_info "  ${PLATFORM}: Bundling ${#LIBS_TO_BUNDLE[@]} libraries"
    
    # Validate each library before bundling
    for lib in "${LIBS_TO_BUNDLE[@]}"; do
        if [[ ! -f "$lib" ]]; then
            log_error "Missing library: $lib for $PLATFORM"
            exit 1
        fi
        log_info "    - $(basename "$lib") ($(lipo -info "$lib" 2>&1))"
    done
    
    libtool -static -o "${FRAMEWORK_DIR}/${FRAMEWORK_NAME}" "${LIBS_TO_BUNDLE[@]}"
    FOUND_ANY=true
else
    log_error "No libraries found for ${BACKEND_NAME} on ${PLATFORM}"
    exit 1  # Fail instead of continuing
fi

Step 3: Verify lipo Fat Binary Creation

Ensure the fat simulator binary combines both architectures correctly:

# Around line 395-398, add verification
lipo -create \
    "${BUILD_DIR}/SIMULATORARM64/${FRAMEWORK_NAME}.framework/${FRAMEWORK_NAME}" \
    "${BUILD_DIR}/SIMULATOR/${FRAMEWORK_NAME}.framework/${FRAMEWORK_NAME}" \
    -output "${SIM_FAT}/${FRAMEWORK_NAME}.framework/${FRAMEWORK_NAME}" 2>/dev/null || true

# Add validation:
if [[ -f "${SIM_FAT}/${FRAMEWORK_NAME}.framework/${FRAMEWORK_NAME}" ]]; then
    log_info "Simulator fat binary architectures: $(lipo -info "${SIM_FAT}/${FRAMEWORK_NAME}.framework/${FRAMEWORK_NAME}")"
else
    log_error "Failed to create simulator fat binary for ${FRAMEWORK_NAME}"
fi

Step 4: Package.swift Linker Settings

Verify simulator-specific linker flags in sdk/runanywhere-swift/Package.swift:

// Around line 169-183, ensure platform-specific settings
linkerSettings: [
    .linkedLibrary("c++"),
    .linkedFramework("Accelerate"),
    .linkedFramework("CoreML"),
    // ... existing settings
]

Consider adding conditional compilation for simulator:

#if targetEnvironment(simulator)
// Simulator-specific initialization
print("[RunAnywhere] Running in Simulator mode")
#endif

4. Implementation Plan

Phase 1: Diagnosis ✅

• Identify the issue (simulator vs device)
• Locate relevant build scripts
• Document current build process

Phase 2: Build Script Improvements 🔄

• Add architecture validation to build-ios.sh
• Improve error logging for missing libraries
• Add lipo -info output for all bundled frameworks
• Make build fail fast if simulator libraries are missing

Files to modify:

• sdk/runanywhere-commons/scripts/build-ios.sh (lines 324-415)

Phase 3: Testing & Verification 🔄

• Clean build: ./scripts/build-ios.sh --clean
• Verify XCFrameworks contain valid simulator slices:

  cd sdk/runanywhere-commons/dist
  for fw in *.xcframework; do
    echo "\n$fw:"
    lipo -info "$fw/ios-arm64_x86_64-simulator/"*.framework/* 2>/dev/null || echo "Missing simulator slice"
  done

• Test sample app on iPhone 16 Pro Simulator
• Test sample app on physical iPhone device
• Test on Intel Mac simulator (if available)

Phase 4: Documentation 📝

• Update sdk/runanywhere-swift/README.md with simulator requirements
• Add troubleshooting section for simulator issues
• Document architecture requirements

5. Success Criteria

✅ Build System:

• Build script validates all libraries before bundling
• Build fails with clear error if simulator libraries are missing
• lipo -info output shows correct architectures: x86_64 arm64

✅ XCFrameworks:

• RACommons.xcframework contains valid simulator slice
• RABackendLLAMACPP.xcframework contains valid simulator slice
• RABackendONNX.xcframework contains valid simulator slice
• All frameworks can be inspected: file, lipo -info, nm commands work

✅ Sample App:

• iOS sample app launches successfully in iPhone Simulator
• System details display correctly in simulator
• LLM text generation works in simulator
• Speech-to-text works in simulator (if supported)
• Text-to-speech works in simulator (if supported)

✅ Testing:

• App works on iPhone 16 Pro Simulator (iOS 17+)
• App works on physical iPhone device
• App works on iPad Simulator
• No runtime crashes or missing symbol errors

Related Issues

None yet - this is a foundational issue for iOS development workflow.

Additional Context

Environment:

• Xcode 15.0+
• iOS 17.0+ deployment target
• macOS 14.0+ for development

Third-party dependencies:

• ONNX Runtime: https://download.onnxruntime.ai/pod-archive-onnxruntime-c-1.17.1.zip
• Sherpa-ONNX: Bundled in third_party/sherpa-onnx-ios/
• llama.cpp: Fetched via CMake

Notes for Implementation

⚠️ When debugging, check:

1. Run xcodebuild -showsdks to see available simulator SDKs
2. Check Xcode build logs for linking errors
3. Use otool -L to inspect binary dependencies
4. Set DYLD_PRINT_LIBRARIES=1 to see runtime loading

Test command:

cd examples/ios/RunAnywhereAI
./scripts/build_and_run.sh simulator "iPhone 16 Pro"