IoT Devices Lab Series 3 - Getting Started with Micropython and the ESP32

Introduction

In this lab, you'll start working with Micropython for the ESP32 at a basic level. You'll work on things like executing code, transferring files, and configuring WiFi connectivity for your Dev Board.

Pre-requisites

Info

This lab guide is written form Mac OS X and Linux. If you have a Windows Machine, go and read the first part of this series to set up a Linux-based IoT Development workstation on a VM on your laptop.

The tutorial assumes you have Python3 and that you have a working environment and dev board as described in the second part of this series. You'll also need the following materials:

• A development environment with direct USB access (your laptop) and Internet connectivity
• WiFi connectivity (user and password based authentication, certificate based authentication will not work). A mobile phone configured to share the connection via WiFi will do.
• An ESP32-based development board
• A USB-A/USB-C to micro USB data cable to connect the dev board to your computer.

Setting up the environment

Connect the Dev Board to your PC using the USB cable, the embedded power LED in the Dev Board should light up.

Now, follow the steps described in second part of this series related to activating the virtual environment and shell environment variables, and fire up your development environment (minicom + ampy, Thonny, VSCode + Pymark... pick your poison).

Basic interaction with Python

We already examined how to get a Micropython REPL using either minicom or Thonny. Let's start first with minicom, open a new shell and launch it:

minicom

You should get the Micropytholn REPL. Now try some very basic code straight in the REPL:

my_group_number = <Your group number>
print('Hello Micropython from group number', my_group_number)

You should the get REPL output through the USB serial connection in your PC.

Let's try now importing the popular machine module for the ESP32:

import machine

machine.freq()          # get the current frequency of the CPU
machine.freq(240000000) # set the CPU frequency to 240 MHz

These two commands get the current frequency of the CPU and set a new CPU frequency to 240 MHz.

Let's see what the impact of the CPU speed is by making a quick test. We'll use this test to also learn how to pass Micropython files to your device.

Open an editor in your computer and type the following Micropython code:

def test_f():
    l = 10000000
    s = 0
    for i in range(l):
        s = s + 1
    return s

Save the file as testing.py. Now exit minicom, open a new shell tab and let's use ampy to pass the file from your computer to the ESP32 device:

ampy put testing.py

and test that the file has been passed by checking the output of:

ampy ls

This function will run a simple loop a number of times performing a simple computation, we'll use as a baseline for our benchmark. Now, let's write the actual benchmarking function, but this time using Thonny. Copy this code into Thonny's IDE main window:

import time
def runtime(func):
     a = time.time()
     b = func()
     print(time.time() - a)
     return b

Click on the save icon, and when presented with the option, select "Micropython device". Name the file benchmarking.py.

To test everything, this time we'll use the REPL that's embedded in Thonny. Go there and type the following commands:

import testing
import benchmarking
import machine
machine.freq(160000000)
benchmarking.runtime(testing.test_f)
machine.freq(240000000)
benchmarking.runtime(testing.test_f)

You should see that, altough increasing power consumption, throttling up the CPU speed does make a difference in the time taken by the simple addition.

We'll now practice putting all this into a file, uploading it to the ESP32 and finally running it there. To use back ampy, you should free up the serial USB connection being used by Thonny by doing Run > Disconnect from Thonny's menu bar before trying these commands:

ampy put src/runtime_test_complete.py
ampy run runtime_test_complete.py

You'll see that ampy throws a timeout that we cannot modify (the timeout is 10 seconds after which pyboard will throw an error). So for testing long running programs like this one, you shouldn't use ampy and should try Thonny instead.

The Micropython filesystem

MicroPython automatically creates a default configuration and auto-detects the primary filesystem present in your device. This filesystem is typically backed by internal flash memory on the device, as is our case with ESP32 DevKitC Dev Board that we're using.

MicroPython implements a Unix-like Virtual File System (VFS) layer. All mounted filesystems are combined into a single virtual filesystem, starting at the root /. Filesystems are mounted into directories in this structure, and at startup the working directory is changed to where the primary filesystem is mounted. On the ESP32 the primary filesystem is mounted at /.

On power-on, MicroPython will attempt to detect the filesystem on the default flash and configure and mount it automatically. If no filesystem is found, MicroPython will attempt to create a FAT filesystem spanning the entire flash. This filesystem is stored in the flash after the MicroPython firmware.

import os
vfs_output = os.statvfs("/")
print(vfs_output)

It will give you a tuple like this:

(4096, 4096, 512, 505, 505, 0, 0, 0, 0, 255)

where the different quantities are (from Micropython's os.statvfs() documentation):

• f_bsize: file system block size
• f_frsize: fragment size
• f_blocks: size of fs in f_frsize units
• f_bfree: number of free blocks
• f_bavail: number of free blocks for unprivileged users
• f_files: number of inodes
• f_ffree: number of free inodes
• f_favail: number of free inodes for unprivileged users
• f_flag: mount flags
• f_namemax: maximum filename length

So, given that we can have a look at how much space we have in our filesystem taking the relevant data from the tuple:

full_size = vfs_output[2]*vfs_output[1]
free_size = vfs_output[3]*vfs_output[1]
megabyte = 1024 * 1024
print(full_size/megabyte, free_size/megabyte)

You should get an output like this:

2.0 1.972656

Accesing the filesystem from inside the REPL is no different to accessing file from Python:

import os
os.listdir()

Try removing some files from the REPL:

rmfiles = ["testing.py","benchmarking.py"]
for file in rmfiles:
    os.remove(file)

The most practical way though to manage files though is through external tools and IDEs like the ones we're using.

Running programs on boot/resets

There are two files that are treated specially by the Micropython when it starts up:

• boot.py: this script is executed first (if it exists) and then once it completes it executes main.py.
• main.py: this script is executed right after boot.py.

You can create these files yourself and populate them with the code that you want to run when the device starts up.

Let's try one very simple boot.py file. Save this content into the file and then press the EN button in the device:

print("\nI'm a Micropython programmer and I'm OK\nI sleep all night, I code all day\n")

Persisting data

The ESP32 has a section called NVM where data can be persisted in flash. However, as MicroPython provides a full filesystem, it is not necessary to use it. We can just persist data in the regular filesystem set up by Micropython.

One way to persist data is storing that data in a Python dictionary that can be serialized in JSON format and stored in flash. Let's understand it better with an example:

import ujson
config = {
  'first': 'Javier',
  'last': 'Canadillas'
}
with open('config.json', 'w') as f:
  ujson.dump(config, f)

You should see the new JSON file in the MicroPython filesystem (if you're using Thonny, just refresh the view in the lower left pane to see it). We can now reload it with the following code:

import ujson
with open('config.json', 'r') as f:
  config = ujson.load(f)
  print(config['last'])

Wireless connectivity

Networking (Wireless) functionality in Micropython is wrapped under the network module. Let's use it to connect our ESP32 Dev Board to an access point.

The first thing you'll need to do here is have an access point with WPA2 protection available. You may configure that in your phone and assume you're calling the WiFi and putting a password .

Let's first play with it from the Micropython REPL:

import network

We first create a station interface (wifi client for connecting to an access point) and set it to active:

wlan = network.WLAN(network.STA_IF)
wlan.active(True)

We can then scan for in-range access points:

wlan.scan()

Your access point should appear in the list. Go ahead and connect to it, passing as the first argument the SSID and as the second argument the WiFi password:

wlan.connect('<your_student_id>', '<your_student_id>')

You should now be connected; you can test the connection and get the assigned IP address:

wlan.isconnected()
wlan.ifconfig() 

You can also configure the ESP32 as an access point, but we won't do it here. More information can be found here.

Connecting to WiFi on boot

If we always want our device to try to connect to WiFi on boot, we can include the following code in our boot.py file (you need to replace down there for the student id you used both as your SSID and your WiFi password):

def do_connect(essid, password):
    import network
    wlan = network.WLAN(network.STA_IF)
    wlan.active(True)
    wlan.config(reconnects=10)
    if not wlan.isconnected():
        print('connecting to network...')
        wlan.connect(essid, password)
        while not wlan.isconnected():
            pass
    print('network config:', wlan.ifconfig())

do_connect('<your_student_id>', '<your_student_id>')

Now, reset your device and check that the device has been successfully connected to the LAN generated by your access point.

Note

The desired configuration is that you register the MAC address of the device in a DHCP server working with the Access Point and configure a fixed IP assignment. That way, the LAN IP address assigned to the ESP32 will always be the same and simplify operations moving forward. You can get the MAC address programmatically from Micropython by doin wlan.config('mac').

Note

On credentials security, be mindful that this approach stores credentials in code, which is a really bad practice for a number of reasons. You could still store the credentials in a json file and modify your do_connect() function so you get the essid and password variables from the returned dictionnary. This way at least the credentials won't get pushed into any code repository if you do it properly:

def get_creds():
    import ujson
    with open(creds.json) as fp:
        data = ujson.load(fp)
        return data

A much more discussion on securing the credentials (like encrypting the flash contents of the device) is out of scope. If you have interest, check out this discussion in the Micropython Official forum.

Accesing the Web REPL

A very good way to actually check that you have web connectivity is to enable Micropython's Web REPL.

WebREPL (REPL over WebSockets, accessible via a web browser) is a feature available in ESP32 port. The daemon listens on all active interfaces, and allows you to connect to the ESP32 via a router (the STA interface) or directly when connected to its access point.

In addition to terminal/command prompt access, WebREPL also has provision for file transfer (both upload and download).

To configure it, just do

import webrepl_setup

and follow on-screen instructions, telling WebREPL to be enabled on boot. My recommendation for you is to use python as password, and then check that the boot.py file contains the following couple of lines that enable WebREPL on boot:

import webrepl
webrepl.start()

Now, open a browser in your computer pointing to the https://micropython.org/webrepl, fill in the IP of the ESP32 board, and provide the password you configured before, and you should get the WebRPL to interact with your board via wireless.

Warning

Remember that you should connect your PC to the same lan where the ESP32 device is connected to. If you're using your mobile as access point, that means that your laptop should connect to that access point as well.

Info

If you don't want to enable WebREPL on boot, you may configure and launch the daemon by hand as follows:

import webrepl
webrepl.start(password='python')

If you're using different networks to connect your computer and ESP32, it'll be more practical moving forward to have the ESP32 device connected to the Internet through the AP of your choosing (like your smartphone's access point) and connect to the ESP32 device using a USB serial connection for development purposes instead of relying on a wireless network for it.

Testing the board with the embedded Hall effect sensor, that is able to

The ESP32 module includes an internal Hall effect sensor, as well as a sensor able to read the internal temperature of the CPU.

import esp32
esp32.hall_sensor()
esp32.raw_temperature()

These sensors are OK for measuring the internal state of the MCU, but are no good for taking external measurements. That's why we'll use external sensors downstream.

Additional Links

• Micropython Quick Reference for the ESP32
• Micropython Quick Reference for the ESP32- Networking
• Espressif API Guide - Flash Encryption
• Micropython forum - How Should WiFi Passwords be securely stored on micropython device?