Enrico's blog

Last 10 blog posts

2016-08-01 00:00:00+02:00

Links for August 2016

First post with the new link collection feature of staticsite!

Heavy Metal and Natural Language Processing [archived]

Natural language processing and Metal lyrics, including the formula for the "metalness" of a word and a list of the most and least metal words.

Confirming all use of an SSH agent [archived]

«For a long time I’ve wanted an ssh-agent setup that would ask me before every use, so I could slightly more comfortably forward authentication over SSH without worrying that my session might get hijacked somewhere at the remote end (I often find myself wanting to pull authenticated git repos on remote hosts). I’m at DebConf this week, which is an ideal time to dig further into these things, so I did so today. As is often the case it turns out this is already possible, if you know how.»

Why We Don’t Report It [archived]

«“Why don’t you report it?” It’s up there on every list I’ve seen of things you shouldn’t say to sexual assault survivors, yet I keep hearing it…»

Voltron, an extensible debugger UI toolkit written in Python

Multi-panel display built from various gdb outputs.

Notmuch, offlineimap and Sieve setup [archived]

Nice description of a notmuch+offlineimap+sieve setup, for when I feel like rethinking my email setup.

Wikipedia:Unusual articles

An endless source of weird and wonderful.

ZERO: no linked HIV transmissions [archived]

«The results provide a dataset to question whether transmission with an undetectable viral load is actually possible. They should help normalise HIV and challenge stigma and discrimination.»

TV pickup

Someone once in the UK told me that it was a big enough problem that so many people turn on their electric kettles during the endtitles of Eastenders, that there's an employee in a hydro plant that needs to watch it to ramp up the power at the right time. I've finally found a wikipedia page about it.

Amazon isn't saying if Echo has been wiretapped [archived]

"We may never know if the feds have hijacked Amazon Echo. … In case you didn't know, Echo is an always-on device, which, when activated, can return search queries, as well as read audiobooks and report sports, traffic, and weather. It can even control smart home devices."

eng pdo
2016-07-09 19:23:35+02:00

Monthly link collections with staticsite

A year ago, I wrote:

Instead of keeping substantial tabs open until I have read all of them, or losing them in the jungle of browser bookmarks, I have written a script that collects them into a file per month, and turns them into markdown files for my blog.

That script turned out to be quirky and overengineered, so much so that I stopped using it myself.

I've now rethought my approach, and downscaled it: instead of saving a copy of each page locally, I can blog a reference to https://archive.org or https://archive.is. I do not need to autogenerate a description from the site itself.

The result has been a nicely minimal set of changes to staticsite that resulted in a new version where adding a link to a monthly collection is as easy as typing ssite new -a links.

As long as I'll remember to rebuild the site 3 weeks from now, a new post should automagically appear in my blog.

eng pdo ssite sw
2016-06-15 21:47:28+02:00

Verifying gpg keys

Suppose you have a gpg keyid like 9F6C6333 that corresponds to both key 1AE0322EB8F74717BDEABF1D44BB1BA79F6C6333 and 88BB08F633073D7129383EE71EA37A0C9F6C6333, and you don't know which of the two to use.

You go to http://pgp.cs.uu.nl/ and find out that the site uses short key IDs, so the two keys are indistinguishable.

Building on Clint's hopenpgp-tools, I made a script that screenscrapes http://pgp.cs.uu.nl/ for trust paths, downloads all the potentially connecting keys in a temporary keyring, and runs hkt findpaths on it:

$ ./verify-trust-paths 1793D6AB75663E6BF104953A634F4BD1E7AD5568 1AE0322EB8F74717BDEABF1D44BB1BA79F6C6333
hkt (hopenpgp-tools) 0.18
Copyright (C) 2012-2016  Clint Adams
hkt comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions.
(4,[1,4,3,6])

(1,1793D6AB75663E6BF104953A634F4BD1E7AD5568)
(3,F8921D3A7404C86E11352215C7197699B29B232A)
(4,C331BA3F75FB723B5873785B06EAA066E397832F)
(6,1AE0322EB8F74717BDEABF1D44BB1BA79F6C6333)

$ ./verify-trust-paths 1793D6AB75663E6BF104953A634F4BD1E7AD5568 88BB08F633073D7129383EE71EA37A0C9F6C6333
hkt (hopenpgp-tools) 0.18
Copyright (C) 2012-2016  Clint Adams
hkt comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions.
(0,[])

This is a start: it could look in the local keyring for all ultimately trusted key finegrprints and use those as starting points. It could just take as an argument a short keyid and automatically check all matching fingerprints.

I'm currently quite busy with https://nm.debian.org and at the moment verify-trust-paths scratches enough of my itch that I can move on with my other things.

Please send patches, or take it over: I'd like to see this grow.

debian eng pdo sw
2016-06-15 10:14:43+02:00

On discomfort and new groups

I recenyly wrote:

When you get involved in a new community, such as Debian, find out early where, if that happens, you can find support, understanding, and help to make it stop.

Last night I asked a group of friends what do they do if they start feeling uncomfortable when they are alone in a group of people they don't know. Here are the replies:

I think a lot of these point are good food for thought about barriers of entry, and about safety nets that a group has or might want to have.

debian eng pdo
2016-06-07 12:43:45+02:00

You'll thank me later

I agree with this post by Matthew Garrett.

I am quite convinced that most of the communities that I have known are vulnerable to people who are good manipulators of people.

Also, in my experience, manipulation by negating, pushing, or reframing the boundaries of people tends not to be recognised as manipulation, let alone abusive behaviour.

It's not about physically forcing people to do things that they don't want to do. It's about pushing people, again and again, wearing them out, making them feel like, despite their actual needs and wants, saying "yes" to you is the only viable way out.

It can happen for sex, and it can happen for getting a patch merged. It can happen out of habit. It can happen for pretty much anything.

Consent culture was not part of my education, and it was something I've had to discover for myself. I assume that to be a common experience, and that pushing against boundaries does happen, even without malicious intentions, on a regular basis.

However, it is not ok.

Take insisting. It is not the same as persisting. Persisting is what I do when I advocate for change. Persisting is what I do when the first version of my code segfaults. Insisting is what I do when a person says "no" to me and I don't want to accept it.

Is it ok to insist that a friend, whom you think is sick, goes and gets help?

Is it ok to insist that a friend, whom you think is sexually repressed, pushes through their boundaries to explore their sexuality with you?

In both cases, one may say, or think, trust me, you'll thank me afterwards. In both cases, what if afterwards I have nothing to thank you for?

I see a common pattern in you'll thank me afterwards situations. It can be in good faith, it can be creepy, it can be abusive, and most of the time, what it is, is dangerously unclear to most of the people involved.

I think that in a community like Debian, at the level of personal interaction, Insisting is not ok.

I think that in a community like Debian, at the level of personal interaction, "You'll thank me afterwards" is not ok.

When I say it's not ok I mean that it should not happen. If it happens, people must be free to say "stop". If it doesn't stop, people must expect to be able to easily find support, understanding, and help to make it stop.

Just like when people upload untested packages.

Pushing against personal boundaries of people is not ok, and pushing against personal boundaries does happen. When you get involved in a new community, such as Debian, find out early where, if that happens, you can find support, understanding, and help to make it stop.

If you cannot find any, or if the only thing you can find is people who say "it never happens here", consider whether you really want to be in that community.

debian eng life pdo
2016-05-31 12:23:31+02:00

Python gotcha

This code:

#!/usr/bin/python3

class Test:
    def __init__(self, items=[]):
        self.items = items

    def add(self, item):
        self.items.append(item)


a = Test()
a.add("foo")
b = Test()
b.add("bar")
print(repr(a.items))
print(repr(b.items))

"obviously" prints:

['foo', 'bar']
['foo', 'bar']

Because the default value of the items argument is a mutable list constructed just once when the code is compiled when the function definition is evaluated, and then reused.

So, in Python, mutable items in default arguments are a good way to get more fun time with debugging.

debian eng pdo
2016-05-23 14:45:34+02:00

I chipped in

I clicked on a random link and I found myself again in front of a wired.com popup that wanted to explain to me what I have to think about adblockers.

This time I was convinced, and I took my wallet out.

I finally donated $35 to AdBlock.

(And then somebody pointed me to uBlock Origin and I switched to that.)

eng pdo rant
2016-03-18 13:05:48+01:00

Adjusting with Moore's Law

In my last post I used zxcvbn to check the strength of passwords, and it gave estimates in hundreds of centuries.

However, there's Moore's law. If we assume that every 18 months the effective computing power will double...

#!/usr/bin/python3

import sys

years = int(sys.argv[1])

# How many 18 months intervals we spent
count = 0
# How many original-computation-time-months we compute each 18 months interval
step = 18
# How many original-computation-time-months we computed so far
total = 0
# How many original-computation-time-months we need
target = years * 12

# Iterate each 18 months interval
while True:
    count += 1
    total += step
    if total > target: break
    step *= 2

print("{:.2f} years needed".format(count / 12 * 18))

...then something that is now estimated in 152 centures is likely to take, well:

$ ./actual_years 15200
9.33 years needed

Update

Fixed algorithm (thanks John MacGregor). Now the estimate is a little longer, but still:

$ ./actual_years 15200
21.00 years needed

Łukasz Stelmach proposed an analytic solution:

There are two cars (remember phisics in primary school?). They both have the same speed V0. It is going to take "54.697 centuries" for car A to travel the desired distance at constant speed. Car B is accelerating, its speed is 2^(t/18m)*V0 (18 [m]onths).

s = V0 * T (car A, T is 54.697 centuries)

s = ∫2^(t/18)*V0 dt (car B, we're looking for t)

It's been some time but as far as I can tell it is

s = V0 * 2^(t/18) * 1/((1/18) * ln 2)

2^(t/18) * 1/((1/18) * ln 2) = 54.697 * 100 * 12 [months]

2^(t/18) = 2527.54

t = 18*log2(2527.54)

t = 16.96 years (still reasonable strength).

debian eng pdo
2016-03-17 16:28:03+01:00

Enhanced xkcd password generator

gpw and xkcdpass have entered testing.

How about putting them together?

#!/usr/bin/python3

import subprocess
import random
import re

re_spaces = re.compile(r"\s+")

def get_words(cmd):
    out = subprocess.check_output(cmd, universal_newlines=True)
    return (x for x in re_spaces.split(out) if x)

words = set()

words.update(get_words(["xkcdpass", "-c", "10"]))

for l in range(6, 9):
    words.update(get_words(["gpw", "10", str(l)]))

for i in range(7):
    print(" ".join(random.sample(words, 5)))

Let's check the results:

#!/usr/bin/python

from __future__ import print_function
from zxcvbn import password_strength
import sys

for line in sys.stdin:
    pw = line.strip()
    info = password_strength(pw)
    print("{pw}: {crack_time_display} {entropy}".format(pw=pw, **info))
$ xkcdpass -n 5 -c 7 | ./checkpass
moribund fasten cologne enamor by-and-by: centuries 132.586
woolen leptonic pogrom roster Serra: centuries 127.145
shorthand Piman suctional elution unholy: centuries 119.809
Blackwell mauvish covey selenium finished: centuries 112.936
asthma bloat crenulate clean-cut cytoplasm: centuries 150.181
agreeable refined racism Captain Cyrenaic: centuries 104.28
Bothnia stagnant atomist these revise: centuries 97.91
$ gpw 7 | ./checkpass
ncenspri: 11.0 days 33.903
orsterma: instant 19.54
entsssel: 14.0 hours 29.788
tempispi: instant 20.058
nidgersi: 14.0 hours 29.725
oligalin: 51.0 minutes 25.824
iseartio: 10.0 minutes 23.207
$ ./metapass | ./checkpass
herups icirop overdub oldster cingsape: centuries 147.265
mujahidin usufruct rition porky prignam: centuries 153.684
cannula irenics tableaux ntshed calcific: centuries 152.196
stopgap stale orthente Amazonia sharer: centuries 113.474
edudives mmingsso eremab prignam tableaux: centuries 147.354
tingei heroism Nathaniel cannula molasses: centuries 103.007
Torres blood lewdness prignam eremab: centuries 118.475

The strength, according to zxcvbn, seems only marginally better, and it's a wonderful way to grow a vocabulary of nonexisting words that can be dropped into cheap talk conversation to confuse the innocents.

debian eng pdo sw
2016-03-16 10:16:12+01:00

Postprocessing files saved by vim

Processing vim buffers through an external program on save, is a technique that is probably in use, and that I have not found documented anywhere, so here we go. Skip to the details for the entertaining example.

I wanted to integrate egt with TaskWarrior so that every time I am editing an egt project file in vim, if I write a line in a certain way then a new taskwarrior task is created. Also, I have never written a vim plugin, and I don't want to dive into another rabbit hole just yet.

Lynoure brought to my attention that taskwiki processes things when the file is saved, so I thought that I can probably get a long way by piping the file from vim into egt and reading it back before saving.

I created a new egt annotate command that reads a project file, tweaks it, and then prints it out; I opened a project file in vim; I typed :%!egt annotate %:p --stdin and saw that it could be done.

I find the result quite fun: I type 15 March: 18:00-21:00 in a log in egt, save the file, and it becomes 15 March: 18:00-21:00 3h. I do something in TaskWarrior, save the file in egt, and the lines that are TaskWarrior tasks update with the new task states.

Details

Here's a step by step example of how to hook into vim in this way.

First thing, create a filter script that processes text files in some way. Let's call this /usr/local/bin/pyvimshell:

#!/usr/bin/python3

import sys
import subprocess

for line in sys.stdin:
    print(line.rstrip(), file=sys.stdout)

line = line.strip()
if line == "$":
    pass
elif line.startswith("$"):
    out = subprocess.check_output(["sh", "-c", line[1:].strip()], universal_newlines=True)
    sys.stdout.write(out)
    if not out.endswith("\n"):
        sys.stdout.write("\n")
    print("$ ", file=sys.stdout)

Then let's create a new filetype in ~/.vim/filetype.vim for our special magic text files:

if exists("did_load_filetypes")
  finish
endif

augroup filetypedetect
  au! BufNewFile,BufRead *.term setf SillyShell
augroup END

If you create a file example.term, open it in vim and type :set ft it should say SillyShell.

Finally, the hook in ~/.vim/after/ftplugin/SillyShell.vim:

function! SillyShellRun()
    :%!/home/enrico/dev/egt/pyvimshell
    :$
endfunction
autocmd BufWritePre,FileWritePre <buffer> :silent call SillyShellRun()

Now you can create a file example.term, open it in vim, type $ ls, save it, and suddenly you have a terminal with infinite scrollback.

For egt, I actually want to preserve my cursor position across saves, and egt also needs to know the path to the project file, so here is the egt version of the hook:

function! EgtAnnotate()
    let l:cur_pos = getpos(".")
    :%!egt annotate --stdin %:p
    call setpos(".", l:cur_pos)
endfunction
autocmd BufWritePre,FileWritePre <buffer> :silent call EgtAnnotate()

I find this adorably dangerous. Have fun.

debian eng pdo sw