Make a list of packages above a given popularity from pkgstats archlinux API

Support
1

I want to make a function to which I give a popularity and I get a list of packages above that popularity from the pkgstats archlinux API. The number of packages it can ask for at a time is limited so I can’t ask for all of them. I can ask for the total number of packages though.

What I’m having trouble with is knowing when I’ve reached the given popularity. What I’ve done right now is download the total number of packages, and then download a few packages at a time keeping just the name for the ones with a popularity above the given one. But I’m downloading all the packages and only keeping some of them, it would be better if I stopped downloading when I reach the given popularity.

#!/usr/bin/env bash

function usage() {
    cat <<EOF
Usage: get_popular_packages.sh popularity
EOF
    exit 1
}

function get_popularity() {
    if [ $# -gt 0 ]; then
        echo $1
    else
        usage
    fi
}

function get_total_packages() {
    curl -sX "GET" -H "accept: application/json" \
        "TheLinkThatMustNotBeNamed api packages?limit=1&offset=0" \
        | jq ".total"
}

function get_packages() {
    curl -sX "GET" -H "accept: application/json" \
        "TheLinkThatMustNotBeNamed api packages?limit=$2&offset=$3" \
         | jq ".packagePopularities[]" \
         | jq "select( .popularity >= $1 )" \
         | jq ".name"
}

function get_popular_packages() {
    local x total limit pkgs
    popularity=$(get_popularity "$@")
    echo "Get packages with a popularity greater than $popularity"
    total=$(get_total_packages)
    echo "There are $total total packages"
    offset=0
    limit=1000
    pkgs=()

    # Get $limit number of packages each time until we reach $total
    while [ $offset -le $total ]; do
        pkgs+=$(get_packages $popularity $limit $offset)
        offset=$(($offset+$limit))
    done

    echo $pkgs
}
2

It’s probably something like this. I still have to test it:

function get_popular_packages() {
    local x total limit pkgs res
    popularity=$(get_popularity "$@")
    echo "Get packages with a popularity greater than $popularity"
    total=$(get_total_packages)
    echo "There are $total total packages"
    offset=0
    limit=200
    res=()
    pkgs=$(get_packages $popularity $limit $offset)

    # Get $limit number of packages
    while [ $offset -le $total ]; do
        pkgs=$(get_packages $popularity $limit $offset)
        res+=$pkgs
        if [ $($pkgs | jq ".packagePopularity[0].popularity") -lt $popularity ]; then
          break
        fi
        offset=$(($offset+$limit))
    done
    echo $res
}
3
4

What I mean by popularity is what you call usage numbers in percent.

5

Housekeeping … no activity …