February 11th:

Software is one of the numerous tasks for today. First, I decided to implement a character input UI element - using the numpad. It won't be as fancy as it's supposed to be, but that's beta. However, there are absolutely crucial features, and all added together they form a bulk of the work.

---

I decided to liberally spray the code with comments, as if whoever will be reading the code will be a Python beginner who has only interacted with the code but hasn't understood its workings yet. It goes against Clean Code, a book I respect and partially live by, but then - the code I'm writing is not for professional coders and there isn't a tightly-knit team of developers, and the code will be read much, much more often than it'll be modified. There's the problem of keeping the comments up to date, but, in general, commenting as the code flows might very likely become a contribution requirement, at least for the software core of the ZeroPhone project (I'm not going to dictate what external apps do, of course, as long as it's not in the core).

Numpad character input UI element is fun. I've discovered that, it seems, there's some kind of bug that makes the display library I'm using kinda crash Python (getting the same error message as this bug in numpy) - if you pass None as a string to be displayed on the display. Here's the commit which has the bug, I incorrectly thought it could've been a bug in the interpreter but then I remembered there actually is an external library I'm using for that and it's what should have the bug. There's a pull request to be sent then, but that's for later =)

There are two threads in any UI element - one is "application's current thread" (ACT, your Python application's thread, where activate() is running), another is "input listener's thread" (ILT, where callbacks attached input events are executed). In this UI element, ILT changes value upon keypresses, and ACT is responsible for "wait a second after last keypress and any consecutive keypresses will make a next character" - anybody who has ever typed an SMS on a keypad phone has encountered this. However, I pressed a button 100 times and got a race condition. Now onto learning threading.Lock.

An hour spent debugging an issue which I myself set up for me to stumble upon. Fail. At least I made this snippet (which didn't help me debug anything, but hey, it prettyprints what currently running threads are doing):

import threading
import traceback
import signal
import sys
def dumpthreads(*args):
    for th in threading.enumerate():
        print(th)
        traceback.print_stack(sys._current_frames()[th.ident])
        print()
signal.signal(signal.SIGUSR1, dumpthreads)

Okay, I might just leave this UI element at "no on-screen pointer, no arrow keys used, thus, no character skipping and in-the-middle editing", at least for today. What's next? An nmap app? Can do.

---

February 12th:

Nope, couldn't. Spent half an hour on "'ImportError: cannot import name IncompleteRead'?" error, then had to go to sleep. "easy_install -U pip" fixed this problem. Let's try to nmap the hackerspace's network using CLI "nmap -sn" first:

There should be at least something that the app I'll write will be able to find. I know most of these devices, but not all, and, given some things there were left after the #Hacker Olympics 2016 @MakeRīga hackerspace, my app should be able to find at least something interesting - something I could show as proof-of-concept.

Let's see the Python-nmap package examples and whip up a simple app skeleton:

try:
    import nmap
except ImportError:
    nmap = None
def callback():
    if nmap is None:
        Printer(ffs("nmap module not found!", o.cols), i, o, 3)
        return False
    menu_contents = [
    ["Scan localhost", scan_localhost]
    #["Scan a network", scan_network],
    #["Scan arbitrary IP", scan_ip],
    ]
def init_app(input, output):
    global i, o
    i = input; o = output

Basically, let's implement "Scan Localhost" function for a start, then go to other functions - as they all have their own requirements, "Scan Network" will need a way to get all available networks and "Scan IP" will need a way to enter an IP using a keypad, localhost scan needs neither and is therefore the easiest for PoC. However, all three will need a "show scan results" function - so we'll implement that in the beginning, too.

def scan_localhost():
    host = "127.0.0.1"
    nm = nmap.PortScanner()
    nm.scan("127.0.0.1", "0-1024")
    show_scan_results()

Hmmm...

def scan_localhost(host = "127.0.0.1"):
    scan_ip(host)
def scan_ip(ip):
    nm = nmap.PortScanner()
    nm.scan(ip, "0-1024")
    show_scan_results_for_ip(nm, ip)

If can lay some groundwork now, why not? Now, showing results:

def show_scan_results_for_ip(nm, ip):
    #Assembling a report we can pass straight to Menu object
    report = []
    #Scan results for the IP, as we get them from nmap interface
    ip_results = nm[ip]
    #IP, state, hostname
    report.append([ "IP: {}, {}".format(ip, ip_results.state()) ])
    report.append([ "Host: {}".format(ip_results.hostname()) ])
    #Now assemble a list of open ports:
    protocols = ip_results.all_protocols()
    if protocols:
        report.append(["Open ports:"])
        for protocol in protocols:
            ports = ip_results[protocol]
            for port in ports:
                report.append(["  {}:{}".format(protocol, port)])
    else: #No open ports for any protocol found?
        report.append(["No open ports found"])
    #Show report
    Menu(report, i, o).activate()

First try, the scanning process actually was so slow I thought it hanged on something and Ctrl+C'ed it. There needs to be some visual feedback:

def scan_ip(ip, ports="0-1023"):
    Printer("Scanning {}:{}".format(ip, ports), i, o, 0) #Leaves the message on the screen
    nm = nmap.PortScanner()
    nm.scan(ip, ports)
    show_scan_results_for_ip(nm, ip)

Should work, let's try again:

I have a feeling I just passed those 20% of work which produce 80% of the result. Okay, now onto scanning networks. I got a library that can get IPs from "ip addr" output (alredy in pyLCI, in "Networks" app), but I need a library to get the network IP from the interface. "ipaddr" library from Google could help, let's see:

def scan_network_by_ip(ip_on_network)
    if ip == "None":
        Printer("No IP to scan!", i, o, 2)
        return False
    network_ip = #Reading library documentation, nothing there but confusing objects
    #...Okay, fuck the ipaddr library, I'll do it myself.

The "if_info" module I'm using to get network interfaces seems to be perfect to stuff some unreadable code into. Some experimenting:

Works. Now, some unreadable byte-represented-as-string manipulation:

The top part is a proof you can write unreadable code in any language. The bottom part shows it works - now gotta finish it off:

It's written in the right way, and hey, who needs tests anyway?

root@zerophone-prototype:~/pyLCI/apps/network_apps/nmap# python -i if_info.py
{'lo': {'addr6': '::1/128', 'ph_addr': None, 'state': 'unknown', 'addr': '127.0.0.1/8'}, 'wlan
0': {'addr6': 'fe80::b1ee:79a9:55db:63f4/64', 'ph_addr': '18:fe:34:02:8d:8e', 'state': 'up', '
addr': '192.168.88.153/24'}}
IP 192.168.88.153/24 is from network 192.168.88.0
>>>

Now, moving on, back to the "scan entire network" thing. However, I'd need to test some code out:

def scan_network_by_ip(ip_on_network):
    if ip_on_network == "None":
        Printer("No IP to scan!", i, o, 2)
        return False
    network_ip = get_network_from_ip(ip_on_network)
    nm = nmap.PortScanner()
    import pdb; pdb.set_trace() #I should probably try Jupyter

Okay, seems like that to scan the network we also need to add the network mask, sounds reasonable. Some more code, small refactorings, bugfixes, and the network scanning feature is ready:

It's all nice, but the menus that you can see can show that I lack one important feature - cursor. The HD44780 displays that pyLCI initially used have a cursor function, this doesn't - it just draws whatever you tell it. It's bad UX and needs to be fixed, and additionally I need to remake the Menu object so that it uses one cursor consistently. Let's see what the display library can offer...

Hmm, I can draw a line, or a rectangle. Should be enough!

Kinda better. Hmm, do I need the arbitrary IP scan thing? I guess not now - just don't feel like validating IPs/writing a keypad layer with proper mapping. Better add "show cursor" support to the numpad UI element - will it be any easy?

Yep, was pretty easy, and fixed a bug on the way there. Guys, overengineering FTW! Let's add some nmap features! First, saving a scan report by pressing a hardware key - from any place. For that, let's make some global storage variables and add an app-wide callback... Not that fast. Say, you go and scan a network (writes to storage), then scan an IP (writes to storage), then come back to network scan menu and press a button to record the scan - and it records the result of the IP, not the network, even though you have long left the IP viewing menu. It's obvious that the results have to be somehow restored, and that's what I'll do - with a decorator.

----------------------------------

February 16th:

Well, I had to go home, spent a day coding (without making a worklog), one day resting and a day sorting out everything related to project, but not to coding. Today, in short, I've finished the nmap app and added an UI element that can input numbers from the numpad - basically, made the NumpadCharInput UI element easily modifiable, so you can easily subclass it and substitute your own mapping. I also added "heuristic scan" and "any IP scanning" functions to the nmap app - aiming to make a video tomorrow morning =)

Some future features for nmap app:

• Caching of results - ain't nobody got time for scanning the same IP twice, takes quite some time so it'll be cool to have in the production version.
• Better support of the nmap flags, in general, just more ways to adjust arguments (submenus?). Also, better support of all the things that appear in the nmap output
• Looking through the captured files - IMO absolutely crucial

---

Some more important things:

1. Making worklogs while coding is cool.
2. Too verbose worklogs aren't cool
3. There has to be some kind of worklog discipline. My thoughts:
   
   1. Not being too verbose - maybe pasting lots of code and big screenshots is excessive. It can be cool for tutorials, but this isn't really a tutorial (though kinda aiming to be one)
   2. Following some kind of timing - I usually use Pomodoro timing when I work, maybe, from that, 5 minutes could be taken to add something to the worklog. That could also help with the verbosity\

That's all for now. Sorry for the chaotic worklog, next one will be more organized - and expect a separate nmap app announce!

#Damn, I should add the photos sometime soon.