• Why is my Python code so slow?!?

    I begrudgingly started using Python when I got my first Raspberry Pi (as PowerShell wasn’t available for the Pi at that point). As a non-developer my development experience was limited to writing PowerShell, with a sprinkling of C#.

    The learning curve wasn’t too bad, I found Automate the Boring Stuff with Python an invaluable resource, along with the associated course on Udemy. I have grown to love Python over the last few years, it’s so flexible and there are a ton of resources out there to help when you get stuck (as I invariably do!) 😕.

    The one thing I love about working in tech is the constant learning – every day is a school day! I learnt last week how to profile code in Python using a simple one-line command courtesy of @Lee_Holmes:

    python -m cProfile -s cumtime 'C:\Profiling.py'
    

    This command inspects a script (in the example above Profiling.py) and outputs details of the cumulative execution time of each function within the script. This is super-useful when trying to pinpoint why a script is running so slowly 🐌.

    I wrote the script below (Profiling.py) which contains three functions which are each called once, each of these functions includes a sleep to simulate their execution time – with function 1 being the quickest and function 3 the slowest. I then ran the profiler on this to see what it reported 🔍.

    You can see in the output below that the profiler reports the cumulative execution time of each function – the script only calls each function once, so this time matches the sleep I included within the function. In the real-world where each function is likely to get called multiple times, you’d have a better view of where time is being spent in execution and identify opportunities to tweak specific functions to reduce the overall execution time.

    I’m sure I’ll be putting this newfound knowledge to good use! 😀.

  • Continuous Integration for Nintendo Game Boy development using GitHub Actions

    I previously posted about writing Hello World for the Nintendo Game Boy using the GBDK.

    I’ve been meaning to spend some quality time with GitHub Actions and found the perfect excuse – doing Continuous Integration for Game Boy development 😀. I bet I can count the people that are interested in this topic in the entire world on one hand! In any case it was a great excuse to learn GitHub Actions 🤖.

    After much trial and error, although admittedly a lot less than I thought! I ended up with the below GitHub Actions workflow, this contains a single job with multiple steps that does the following:

    • Triggers when code is pushed to the main branch of the repo
    • Checks out the code in the repo
    • Downloads a copy of GBDK from https://github.com/gbdk-2020/gbdk-2020/releases/latest/download/gbdk-win.zip and extracts the contents to the Windows based runner for the Action (this is required to compile the code)
    • Uses lcc.exe (from the GBDK) to build the C source file (main.c) into a Game Boy ROM (helloworld.gb)
    • Creates a release using a random number (generated using PowerShell) for the tag and release name
    • Uploads the helloworld.gb ROM to the release
    name: Game Boy CI
    # Controls when the workflow will run
    on:
      # Triggers the workflow on push
      push:
        branches: [ main ]
      # Allows you to run this workflow manually from the Actions tab
      workflow_dispatch:
    jobs:
      # This workflow contains a single job called "build"
      build:
        # The type of runner that the job will run on
        runs-on: windows-latest
        steps:
          # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
          - uses: actions/checkout@v2
          - name: Compile C source code into .GB ROM file
            run: |
              Invoke-WebRequest -Uri https://github.com/gbdk-2020/gbdk-2020/releases/latest/download/gbdk-win.zip -Outfile GBDK.zip
              Expand-Archive GBDK.zip -DestinationPath GBDK
              ./"GBDK\gbdk\bin\lcc.exe" -o helloworld.gb main.c
              echo "RANDOM=$(Get-Random)" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf-8 -Append
          - name: Create Release
            id: create_release
            uses: actions/create-release@v1
            env:
              GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # This token is provided by Actions, you do not need to create your own token
            with:
              tag_name: ${{env.RANDOM}}
              release_name: Release ${{env.RANDOM}}
              draft: false
              prerelease: false
          - name: Upload Release Asset
            id: upload-release-asset 
            uses: actions/upload-release-asset@v1
            env:
              GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
            with:
              upload_url: ${{ steps.create_release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps 
              asset_path: helloworld.gb
              asset_name: helloworld.gb
              asset_content_type: application/octet-stream
    

    Here’s the Action in “action”

    …and here is the result, a freshly compiled GB ROM.

    Here is a direct link to the repo, if you’d like a closer look.

  • Writing “Hello World” for a Nintendo Game Boy!

    I am a huge retro-gaming geek, I love reliving my childhood and playing classics such as Super Mario Land and Donkey Kong. The Nintendo Game Boy was (and still is!) one of my favourite systems. I’ve always stayed clear of doing any development on retro-gaming systems from the 80s/90s though as this typically involves writing code in assembly language and the learning curve for a non-dev like me is far too high 🧠.

    I recently discovered the Game Boy Development Kit (GBDK) on GitHub, which allows you to write software for the Nintendo Game Boy (and a few other systems too!) in C.

    I’m certainly no expert in C, however I was fairly sure that I could knock up “Hello World” without too much of an issue.

    I downloaded the GBDK and extracted this to a folder (no install is required), I then set about writing this masterpiece –

    #include <stdio.h>
    
    void main()
    {
    int counter = 1;
    
    while (counter <=16)
        {
        printf("\nHello World!");
        counter++;
        }
    }
    

    This is similar to the BASIC – 10 Print “Hello World”, 20 GOTO 10 that fills the screen with “Hello World” that I first wrote on my Amstrad CPC 6128 back in 1990 😀.

    Once I’d saved the file (naming this “helloworld.c“), I then compiled the code using the following command. This creates a ROM for the Game Boy named helloworld.gb

    ./gbdk\bin\lcc.exe -o helloworld.gb helloworld.c
    
    Once the code had compiled, which literally took a second ⏱️. I then headed over to https://virtualconsoles.com/online-emulators/gameboy/ which is an online emulator for the Game Boy and loaded up my newly created ROM.

    Voila!

    12-year-old me would have been amazed!

  • Inspecting Azure Function logs from the command line 🔎

    I’ve been playing around with Azure Functions recently. One thing I like about them is how you can code, test and view the logs real-time from within the Azure Portal – below you can see me testing my Burger Tax function!

    One thing I was interested in doing is getting access to the logs remotely (directly from my machine rather than going through the Azure Portal). It turns out that you can do this using the Azure Functions Core Tools.

    I installed the Core Tools (the Azure CLI is a pre-requisite) and after logging into Azure using az login. I could connect to the logs using this command:

    func azure functionapp logstream <FunctionAppName>
    

    In the example below, I connected to an Azure Function App named burgertax26343 (A very catchy name I know!). To over-complicate things I ran this within Ubuntu running on WSL on my Windows 11 device – you can of course run this natively on Windows, macOS and Linux.

    I then fired up PowerShell to send a test request to the Azure Function App using Invoke-RestMethod (this example is a Starling Bank webhook, read more about how I’m using this here).

    After running this, I flipped back to the other terminal window, where I could see the output – in this case it used Write-Host to confirm that the function had triggered and outputs the name of the merchant (Next in this case).

  • How an Azure Function is keeping me healthy! 🏃‍♂️

    You may have heard of the Sugar Tax……introducing the Burger Tax! 🍔💷.

    I’ve created a solution using the Starling Bank developer API and an Azure Function that “taxes” me whenever I buy junk food (specifically McDonalds!), moving 20% of the transaction into a Savings Space within my Starling Bank account.

    I’ve put together a video that walks through the solution end-to-end.

    The code for the Azure Function is available here.

    Other useful resources:

    Enjoy!

  • Automating Azure Function App creation with the Azure CLI

    I’m currently in the process of writing an Azure Function that I’ll be using with a Starling Bank webhook to “tax” myself every time I purchase junk food…..more on that in a future post though!

    I love automating things and that coupled with getting bored of using the Azure Portal led me to taking a closer look at the Azure CLI, to automate the creation and configuration of the Function App.

    The Azure CLI can be installed on Windows, macOS and Linux. I installed it on Ubuntu which runs on my Windows 11 device using Windows Subsystem for Linux (WSL). I wanted to experience it on a non-Windows platform, which is why I used the convoluted approach of running it on Linux on Windows 😀. Installation was straightforward and required a single command:

    curl -L https://aka.ms/InstallAzureCli | bash
    

    My aim was to use the Azure CLI to create an Azure Function App that runs on Windows with the PowerShell runtime based in the UK South region, I also wanted to add a key/value pair to the Application Settings which I will use to store my Personal Access Token from Starling. The PAT will be used to connect to my bank account and “tax” fast food purchases! I could/should have used Azure Key Vault for this but didn’t want to introduce extra complexity into a hobby project.

    After logging into my Azure subscription using az login from Ubuntu I ran the following to declare the appname and region variables. I’m lazy and use appname for the Resource Group, Storage Account and Function App name. I used the Bash function $RANDOM to append a random number to the app name, which was useful during testing, so I didn’t have to update the app name manually after each run of the script (and there were many as I got to grips with the syntax of the Azure CLI!)

    appname=burgertax$RANDOM
    region=uksouth
    

    I then created a Resource Group to house the Function App, located in the UK South region and named burgertax (appended with a random number).

    az group create --name $appname --location $region
    

    Once the Resource Group had been created, I created a Storage Account which is used to host the content of the Function App.

    az storage account create \
      --name $appname \
      --location $region \
      --resource-group $appname \
      --sku Standard_LRS
    

    Now that I had a Resource Group and Storage Account, I could create the Function App.

    az functionapp create \
      --name $appname \
      --storage-account $appname \
      --consumption-plan-location $region \
      --resource-group $appname \
      --os-type Windows \
      --runtime powershell \
      --functions-version 3
    

    Once this had completed, I ran the following to add a key/value to the Application Settings of the Function App

    az functionapp config appsettings set --name $appname --resource-group $appname --settings "MyKey=MyValue"
    

    …and this to verify that it had created the Application Setting.

    az functionapp config appsettings list --name $appname --resource-group $appname
    

    Lastly, I fired up the Azure Portal to admire my automation.

    Now the fun part……writing the code for my “Burger Tax” solution 🍔.

  • Inspecting webhook calls with ngrok

    I’ve been experimenting with the Starling Bank developer APIs, which I wrote about in Get access to my bank account using Python….why not?!? and Creating an Alexa skill to read my bank balance.

    The next thing on my list to experiment with was creating a webhook and getting this to call an Azure Function to “do something”………which I’m yet to decide exactly what 😀. It’s possible to create a webhook to fire for three specific event types (or any combination of the three) with Starling Bank:

    • Feed Item (for example making a purchase)
    • Standing Order (creating/editing a standing order)
    • Standing Order Payment

    Before I started jumping into creating an Azure Function to “do something” based upon a webhook call from my Starling Bank account, I needed to see what these calls look like, so that I could figure out the data I had to play with. This is where I had the bright idea of using ngrok. My theory was that I could publish an endpoint on my local machine, publish this to the Internet and configure the Starling Bank webhook to call this public URL and then inspect the request made. I’ve previously written about ngrok here.

    This is how I did it:

    Step 1 – Configure ngrok

    I downloaded the ngrok agent from here and copied it to my Raspberry Pi (you can also run on Windows, macOS and other variants of Linux).

    I also created a free account with ngrok as this unlocks some additional features, this provides an Authtoken to authenticate your ngrok agent.

    I then ran the following to register my Authtoken from the directory that ngrok had been copied to (this isn’t my Authtoken BTW)

    Once that completed, I then created a tunnel:
    ngrok http 80
    

    Once this command had run, it provided the external URLs that the local port (80) has been exposed to (by default it will create HTTP and HTTPS endpoint if the command above is run). It’s then as simple as configuring the webhook to call the HTTPS endpoint that has been exposed.

    Step 2 – Configure the Starling Bank Webhook

    I logged into my developer account and created the webhook, I called this ngrok and in the Payload URL entered the public endpoint HTTPS exposed by ngrok. I selected Feed Item as I’m only interested in data generated by purchases and then hit Create V2 Webhook.

    Step 3 – Create a Transaction

    To get the webhook to fire so that I could inspect it using ngrok I needed to spend some money 💷! I headed over to Amazon and bought something using my Starling Bank account.

    Step 4 – Inspecting the Webhook Call

    I then flipped back over to my Raspberry Pi and launched the Web Interface for ngrok – which by default is http://127.0.0.1:4040.

    From here I could see the call made by the webhook, this had failed because I didn’t have anything listening on port 80, which is fine – all I wanted was to inspect the webhook call to see the JSON included. Which I could in all its glory:

    Now that I know what is included, I’m going to create an Azure Function that the webhook will call (instead of the current ngrok black hole) to do something with the transaction data……I am tempted to “tax” myself when I buy fast food by transferring a percentage of the transaction to a savings “space” within my account. I should be able to use the counterPartyName and amount to do this.

  • Creating an Alexa skill to read my bank balance

    I recently posted about my experiences with the Starling Bank developer API’s and shared some Python that I’d written to retrieve my bank balance and most recent transaction.

    I’ve played around with creating Alexa skills before……my most useful was a skill that told me when my rubbish (garbage) 🗑️ was next going to be collected and which colour bin I needed to put out, a real lifesaver!

    I decided to have a go at creating an Alexa skill that leveraged the code I’d written to check my bank balance and retrieve my most recent transaction. It was a success and here it is in action!

    Rather than write a step-by-step blog post that describes how to do it, I thought I’d put together a walkthrough video, here it is in all its glory.

    The AWS Lambda function used by the Alexa Skill (that I demo’d in the video) can be found on GitHub.

  • Get access to my bank account using Python….why not?!? 🐍

    I recently opened an account with Starling Bank, they are a UK based challenger bank. I had read “Banking On It: How I Disrupted an Industry” by their founder Anne Boden (a fantastic book BTW!) and was really intrigued as to how they would stack up against my current “traditional” bank…..anyway, this isn’t a blog about finance! I noticed that they had an API available, which was the perfect excuse for me to geek out 😀.

    It was straightforward to get started, all I had to do was create a developer account and link this to my Starling bank account, which took me less than 5 minutes. Once I’d done this I headed over to the “Getting Started” page within their developer site and create a Personal Access Token (PAT) that could be used to access my account – as I wasn’t planning to create any apps that access anything other than my own personal account, I didn’t need to bother registering an app. When creating a PAT you can specify the scope – for example you may only want the PAT to be able to read your balance (wouldn’t it be nice if you could write to the balance 💵???).

    As I wasn’t entirely sure what I was going to do with this newfound access to my bank account, I selected everything.

    The API and documentation provided by Starling is superb, there’s even a Sandbox to play around in. I played around with the API for a couple of hours and managed to create some Python code to print my balance and details of my most recent transaction.

    I put together the Python code below, which does the following:

    • Imports the required modules – requests is used to make the calls to the API and datetime is used to calculate dates within the get_transactions() function.
    • Stores the Personal Access Token in the PAT variable (breaking every single security best practice in one fell swoop)
    • Defines four functions:
      • get_account() – Retrieves the unique identifier of a bank account, as a user can have more than one account it’s important that the correct account is selected. As I have a single account, I just return what’s at index 0. This function is used by the get_balance() and get_transactions() functions.
      • get_default_category() – This retrieves the default category for an account, this is required for the get_transactions() function. An account can have multiple categories defined; new categories are created when additional spaces are added to an account. A space is akin to a virtual account within an account, for example you could create a space specifically for holiday savings – a really cool feature and something that was new to me.
      • get_balance() Returns the current balance of the account – I used the “effectiveBalance” which considers pending transactions so is more accurate than the alternative “clearedBalance”, which does not.
      • get_transactions()Returns details of the most recent transaction within a specific date range, this takes a single argument (days). Which is the number of days from today to look backwards.
    import requests
    import datetime
    PAT = "insert PAT here"
    url = "https://api.starlingbank.com/api/v2/"
    headers = {"Authorization": "Bearer " + PAT}
    def get_account():
        r = requests.get(url + "accounts", headers=headers)
        return r.json()["accounts"][0]["accountUid"]
    def get_default_category():
        r = requests.get(url + "accounts", headers=headers)
        return r.json()["accounts"][0]["defaultCategory"]
    def get_balance():
        balance = requests.get(url + "accounts/" + (get_account()) + "/balance", headers=headers)
        print(str("£") + str(balance.json()["effectiveBalance"]["minorUnits"] / 100))
    def get_transactions(days):
        datefrom = (datetime.datetime.now()-datetime.timedelta(days=days)).strftime("%Y-%m-%d") + "T00:00:00Z"
        feeditems = requests.get(url + "feed/account/" + (get_account()) + "/category/" + (get_default_category()) + "?changesSince=" + datefrom, headers=headers)
        print(feeditems.json()["feedItems"][0]["source"] + "\n" + feeditems.json()["feedItems"][0]["direction"] + "\n" + feeditems.json()["feedItems"][0]["amount"]["currency"] \
        + ":" + str(feeditems.json()["feedItems"][0]["amount"]["minorUnits"] / 100))
    

    I’ve upload this script to GitHub too.

    Here’s the functions in action! The call to get_balance demonstrates how rich I am! For get_transactions() I passed the argument 30, which will return the most recent transaction in the previous 30 days. Including who the transaction was with, the direction (money in/out) and the amount.

    As I continue to experiment with the API, I’m tempted to write an Alexa skill that I can use to query my account 🤔.

  • Going back to the 1980s with a Raspberry Pi Pico

    In my continued mission to purchase every device that the Raspberry Pi Foundation releases, I acquired a Raspberry Pi Pico shortly before the holiday season 😀.

    Pi Pico

    The Pico is a super-cheap (£3.60) microcontroller that is programmable with MicroPython (a scaled down version of Python) and C, you can find out more about it here.

    I splashed the cash and spent £11 on a Maker Pi Pico, which is a Pico that is pre-soldered onto a maker board that has an LED indicator for each GPIO pin, 3 programmable pushbuttons, an RGB LED, buzzer, stereo 3.5mm audio jack, micro-SD card slot, ESP-01 socket and 6 Grove ports.

    Maker Pi Pico

    To program the device using MicroPython you need to install MicroPython on the Pico (full instructions here) and then install the Thonny Python IDE on the device you’d like to do development on (Windows, Mac or Linux), I may add that if you wanted to do development on a Raspberry Pi and you are running Raspberry Pi OS, this step isn’t required as Thonny comes pre-installed. A step-by-step guide on how to use Thonny with a Pico is available here.

    The biggest mistake I made was to use a cheap USB cable that didn’t support data, I was scratching my head for a while figuring out why my Windows 11 machine couldn’t see the Pico (it presents itself as a mass storage device) until I tried another cable, which worked like a charm.

    After playing around with some of the sample Python scripts for the Maker Pi Pico, I thought I’d go all 1980s and try to re-create a super-basic graphic equalizer. If you don’t have a clue what I’m talking about check out this YouTube video for a demo – https://www.youtube.com/watch?v=GlgVoYH6bPo.

    I put together the script below (also available on GitHub here). Which does the following:

    1. Turns off the LEDs attached to the GPIO pins 0-15
    2. Generates a random number between 0-15
    3. Lights up each LED in order e.g. if 4 is the random number generated it will turn on the LED for GPIO 0, 1, 2, 3 and then 4
    4. Turns off each LED in reverse order, therefore 4, 3, 2, 1 and then 0
    5. Repeats step 2

    I’m sure that there are far more practical applications for a Pico, but this kept me amused for a short while.

    import machine
    import utime
    import random
    
    for i in range(15):
        machine.Pin(i,machine.Pin.OUT)
    
    for i in range(15):
        machine.Pin(i).value(0)
    
    def lightup(id):
        for i in range(id):           
            machine.Pin(i).value(1)
            utime.sleep(0.01)
            
        for i in reversed(range(id)):           
            machine.Pin(i).value(0)
            utime.sleep(0.01)
            
    while True:
        randnum = random.randint(0,15)
        lightup(randnum)
        utime.sleep(0.15)  
    

    Here it is in all its glory: