Uploading a file with progress in Kotlin

Do you want to upload a file using the clean Retrofit syntax, but aren’t sure how to receive the result as well as the upload progress? We will be using Retrofit to perform the file upload, building an implementation that is able to receive the completion progress at intervals and then complete with the remote API response.

Whilst long-running operations are happening, it is nice for the user to see that activity is occurring, such as a progress view being displayed. For the case of a file upload we can show the real progress, which can be represented by the number of bytes transmitted out of the total file size.

We will use the APIs available to us in Retrofit , OkHttp and Okio to build a class that can be used whenever we want a request to publish its progress to whoever wishes to observe it! :up:

Endpoint

We are developing a messaging application that is able to attach a file to a message thread. It is worth noting that the reactive component uses RxJava, however, it can be altered to use regular callbacks or Kotlin Coroutines and suspend functions.

Our endpoint is a POST request that contains a multipart body, consisting of the filename, file MIME type, file size and the file itself. We can define it using Retrofit, specifying the required parts.

@Multipart
@POST("file")
fun attachFile(
    @Part("name") filename: RequestBody,
    @Part("type") mimeType: RequestBody,
    @Part("size") fileSize: RequestBody,
    @Part filePart: MultipartBody.Part
): Single<AttachmentUploadedRemoteDto>

Counting progress

If we just wanted to upload the file without any progress, we would simply convert the file to a request body and send it in the request.

fun createUploadRequestBody(file: File, mimeType: String) = 
    file.asRequestBody(mimeType.toMediaType())

Monitoring upload progress can be achieved by using our own CountingRequestBody which wraps around the file RequestBody that would have been used before. The data that is transmitted is the same as before, allowing the raw file RequestBody to be delegated to for the content type and content length.

class CountingRequestBody(
    private val requestBody: RequestBody,
    private val onProgressUpdate: CountingRequestListener
) : RequestBody() {
    override fun contentType() = requestBody.contentType()

    @Throws(IOException::class)
    override fun contentLength() = requestBody.contentLength()

    ...
}

Transmitting the request body is performed by writing it to a Sink , we will wrap the default sink with our own one that counts the bytes that are transmitted and reports them back via a progress callback.

typealias CountingRequestListener = (bytesWritten: Long, contentLength: Long) -> Unit

class CountingSink(
    sink: Sink,
    private val requestBody: RequestBody,
    private val onProgressUpdate: CountingRequestListener
) : ForwardingSink(sink) {
    private var bytesWritten = 0L

    override fun write(source: Buffer, byteCount: Long) {
        super.write(source, byteCount)

        bytesWritten += byteCount
        onProgressUpdate(bytesWritten, requestBody.contentLength())
    }
}

Within CountingRequestBody we can wrap the default sink into our new CountingSink and write to a buffered version of that, in order to both transmit the file and observe its progress. :eyes:

class CountingRequestBody(...) : RequestBody() {
    ...

    @Throws(IOException::class)
    override fun writeTo(sink: BufferedSink) {
        val countingSink = CountingSink(sink, this, onProgressUpdate)
        val bufferedSink = countingSink.buffer()

        requestBody.writeTo(bufferedSink)

        bufferedSink.flush()
    }
}

The result

Whilst observing the upload progress, there will either be progress or a completed response, the perfect candidate for a sealed class. This will allow CountingRequestResult to be the return type and callers can handle both progress updates and the completed result.

sealed class CountingRequestResult<ResultT> {
    data class Progress<ResultT>(
        val progressFraction: Double
    ) : CountingRequestResult<ResultT>()

    data class Completed<ResultT>(
        val result: ResultT
    ) : CountingRequestResult<ResultT>()
}

Perform the upload

Now that we have a way of uploading a file and receiving the upload progress, we can write our FileUploader . Creating the request body for our upload request involves using a CountingRequestBody that reports progress and completion to a PublishSubject (or another reactive type).

private fun createUploadRequestBody(
    file: File,
    mimeType: String,
    progressEmitter: PublishSubject<Double>
): RequestBody {
    val fileRequestBody = file.asRequestBody(mimeType.toMediaType())
    return CountingRequestBody(fileRequestBody) { bytesWritten, contentLength ->
        val progress = 1.0 * bytesWritten / contentLength
        progressEmitter.onNext(progress)

        if (progress >= 1.0) {
            progressEmitter.onComplete()
        }
    }
}

The upload request consists of using the Retrofit function we implemented at the beginning, providing the file details and the created request body that will count progress. The Retrofit definition and the format of the request parts will depend on how each particular API is put together. Here we are using a request that contains various plaintext parts for the file details and then one for the file to be uploaded.

private fun createUploadRequest(
    filename: String,
    file: File,
    mimeType: String,
    progressEmitter: PublishSubject<Double>
): Single<AttachmentUploadedRemoteDto> {
    val requestBody = createUploadRequestBody(file, mimeType, progressEmitter)
    return remoteApi.attachFile(
        filename = filename.toPlainTextBody(),
        mimeType = mimeType.toPlainTextBody(),
        fileSize = file.length().toString().toPlainTextBody(),
        filePart = MultipartBody.Part.createFormData(
            name = "files[]",
            filename = filename,
            body = requestBody
        )
    )
}

private fun String.toPlainTextBody() = toRequestBody("text/plain".toMediaType())

Our main upload function can put together all of these parts to create a single result stream. We will be able to observe this to get progress updates as well as the final result.

fun uploadAttachment(
    filename: String, file: File, mimeType: String
): Observable<AttachmentUploadRemoteResult> {
    val progressEmitter = PublishSubject.create<Double>()
    val uploadRequest = createUploadRequest(
        filename, file, mimeType, progressEmitter
    )

    val uploadResult = uploadRequest
        .map<AttachmentUploadRemoteResult> { 
            CountingRequestResult.Completed(it.result) 
        }
        .toObservable()

    val progressResult = progressEmitter
        .map<AttachmentUploadRemoteResult> { 
            CountingRequestResult.Progress(it) 
        }

    return progressResult.mergeWith(uploadResult)
}

typealias AttachmentUploadRemoteResult = 
    CountingRequestResult<AttachmentUploadedRemoteDto>

We can now upload a file to our API and update a view as the request progresses, which is nice for noticeably long operations like uploading larger files.

uploader.uploadAttachment(request.filename, request.file, request.mimeType)
    .subscribeOn(appRxSchedulers.io)
    .observeOn(appRxSchedulers.main)
    .subscribeBy(
        onError = { error ->
            // Display error alert
        },
        onComplete = {
            // Display completed Snackbar
        },
        onNext = { progress ->
            // Update progress bar
        }
    )
    .addTo(disposeBag)

Wrap up

Monitoring the progress of a web request may not be immediately obvious when reading through the Retrofit API, however, the powerful APIs of OkHttp and Okio can get the job done nicely. The solution we have developed can be used for any web request, as the counting process can be wrapped around any RequestBody that needs to be sent in a request.

Do you have any requests in your apps that could benefit from observing their progress? If you already have a solution do you use something similar or do you have a different way of doing it? Please feel free to put forward any thoughts or questions you have on Twitter @lordcodes .

If you like what you have read, please don’t hesitate to share the article andsubscribe to my feed if you are interested.

Thanks for reading and happy coding! :pray: