How to get Hacker News top stories using parallel coroutine and Retrofit

Issue #379

interface Api {
    @GET("topstories.json?print=pretty")
    suspend fun getTopStories(): List<Int>

    @GET("item/{id}.json?print=pretty")
    suspend fun getStory(@Path("id") id: Int): Item
}

class Repo {
    fun api(): Api {
        return Retrofit.Builder()
            .baseUrl("https://hacker-news.firebaseio.com/v0/")
            .addConverterFactory(MoshiConverterFactory.create())
            .build()
            .create(Api::class.java)
    }
}

class ViewModel(val repo: Repo): ViewModel() {
    val items = MutableLiveData<ArrayList<Item>>()

    suspend fun load() {
        try {
            val ids = repo.api()
                .getTopStories()
                .take(20)

            val items = ids.map {
                repo.api().getStory(it)
            }
            this.items.value = items.toCollection(ArrayList())
        } catch (e: Exception) {
            this.items.value = arrayListOf()
        }
    }
}

Running parallel

The above run in serial. To run in parallel, we can use async

import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope

class ViewModel(val repo: Repo): ViewModel() {
    val items = MutableLiveData<ArrayList<Item>>()

    suspend fun load() {
        try {
            val ids = repo.api()
                .getTopStories()
                .take(20)

            coroutineScope {
                val items = ids
                    .map { async { repo.api().getStory(it) } }
                    .awaitAll()

                this@ViewModel.items.value = items.toCollection(ArrayList())
            }

        } catch (e: Exception) {
            this.items.value = arrayListOf()
        }
    }
}

Parallel decomposition

https://medium.com/@elizarov/structured-concurrency-722d765aa952

With structured concurrency async coroutine builder became an extension on CoroutineScope just like launch did. You cannot simply write async { … } anymore, you have to provide a scope. A proper example of parallel decomposition becomes:

coroutineScope

https://proandroiddev.com/part-2-coroutine-cancellation-and-structured-concurrency-2dbc6583c07d

coroutineScope function can be used to create a custom scope that suspends and only completes when all coroutines launched within that scope complete. If any of the children coroutines within the coroutineScope throws an exception, all other running sibling coroutines gets cancelled and this exception is propagated up the hierarchy. If the parent coroutine at the top of the hierarchy does not handle this error, it will also be cancelled.

awaitAll

https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/await-all.html

Awaits for completion of given deferred values without blocking a thread and resumes normally with the list of values when all deferred computations are complete or resumes with the first thrown exception if any of computations complete exceptionally including cancellation.

This function is not equivalent to deferreds.map { it.await() } which fails only when it sequentially gets to wait for the failing deferred, while this awaitAll fails immediately as soon as any of the deferreds fail.

This suspending function is cancellable. If the Job of the current coroutine is cancelled or completed while this suspending function is waiting, this function immediately resumes with CancellationException.

Read more

• https://jivimberg.io/blog/2018/05/04/parallel-map-in-kotlin/
• https://stackoverflow.com/questions/53535977/coroutines-runblocking-vs-coroutinescope
• awaitAll https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/await-all.html