Making our pageview data easily accessible

http://blog.wikimedia.org/2015/12/14/pageview-data-easily-accessible/

by devhxinc • 11 years ago

29 2 11 years ago

blog.wikimedia.org

danso• 11 years ago

edit: Added the use of `date` to automatically return last 120 days of data, and crudely label the axes

edit: Here's a gist https://gist.github.com/dannguyen/f415b1797f686f995f8e

I thought there's a Bash one-liner in there somewhere to read the API and convert it to a Google Static chart...but I haven't had my full cup of coffee this morning. Also, my Bash-fu is weak, so here it is as a partially completed function...I'm sure someone here can turn it into a one-liner:

Given a user argument, such as "Star_Wars" [0], the JSON response is parsed with the jq tool [1], then wrangled into a series of comma-delimited values to be passed into Google Static Charts API [2]:

Resulting URL which is a timeseries of the past 120 days of daily pageviews for "Star_Wars":

https://chart.googleapis.com/chart?chs=800x200&cht=bvg&chd=t...

Here's the bash function

    function chart_wpgviews(){
      # first argument is article name
      # eh, too lazy to look up string comparison, so hardcoding the start and end date vals, 
      # since it seems to be locked at a max of 4 months anyway...and hard-coding en.wikipedia
      PAGENAME="$1"
      PROJECT="en.wikipedia"
      ENDTIME=$(date +%s) 
      STARTTIME=$(date --date="@$(($ENDTIME-120*24*60*60))" +%s) # 120 days ago
      ENDDATE=$(date --date="@$ENDTIME" +%Y-%m-%d)
      STARTDATE=$(date --date="@$STARTTIME" +%Y-%m-%d)

      RESPONSE=$(curl -s https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/$PROJECT/all-access/user/$PAGENAME/daily/$(tr -d '-' <<< $STARTDATE)/$(tr -d '-' <<< $ENDDATE))
      VALS=$(jq '.items[] | .views' <<< $RESPONSE | tr '\n' ',')
      MAXVAL=$(jq '.items[] | .views' <<< $RESPONSE | sort -rn | head -n1)
      echo "https://chart.googleapis.com/chart?chs=800x200&cht=bvg&chd=t:${VALS%?}&chds=0,$MAXVAL&chbh=5,0,1&chxt=x,y&chxp=0,0&chxl=0:|$STARTDATE|$ENDDATE|1:||$((MAXVAL/2))|$MAXVAL&chtt=Last+120+days+of+pageviews+for+Wikipedia+page+on+$PAGENAME"
    }

[0] https://wikimedia.org/api/rest_v1/metrics/pageviews/per-arti...

[1] https://stedolan.github.io/jq/

[2] https://developers.google.com/chart/image/docs/data_formats

flashman• 11 years ago

http://top.hatnote.com/

Wow, it's like a window into the zeitgeist.

Making our pageview data easily accessible

Related Stories