Close

Hack Chat Transcript, Part 1

A event log for Inside Smart Meters Hack Chat

Are you smarter than your electrical meter?

Dan MaloneyDan Maloney 04/14/2021 at 20:030 Comments

OK, folks, here we go! Welcome to the Hack Chat, I'm Dan and I'll be moderating today along with Dusan as we welcome Hash to the Hack Chat for a discussion on smart meters. Really looking forward to this one!

Hash, I saw you one before, you still out there? If so, can you tell us a little about how you got interested in meter hacking?

felix1063 joined the room.12:00 PM

Todd Christell12:00 PM
It also lets them evaluate electrical outages and prioritize their dispatch -- if you're looking for a positive.

Darrin B joined the room.12:00 PM

Hash12:00 PM
Hey everyone!

Maxwell Faraday joined the room.12:00 PM

And just for the record, when we say "meter hacking", we're not talking about anything illegal -- just listening in on meter comms.

James Murphy12:01 PM
Hey Hash!

Hash12:01 PM
Exactly

Hash12:01 PM
I was always interested in hacking and using devices for reasons other than their intended purpose. It’s like a game between me and the device, a puzzle with an unknown number of pieces and no box with an image showing you how it should look when you are done. The prize is the feeling humans have been searching for since the beginning of time: Discovering new places no one else has been.

2rkoester joined the room.12:01 PM

Hash12:01 PM
Now I can afford a nice lab setup so in my spare time I hack for fun, hardware and RF interest me most and I program when needed towards those ends. Power meters caught my eye initially because they get deployed and then basically stay the same for 15 years! This allows me to leisurely hack them knowing what I learn won’t be obsolete in 6 months like with consumer goods.

a.mordicus joined the room.12:02 PM

dbcorbin joined the room.12:02 PM

Hash12:02 PM

Edmund joined the room.12:02 PM

Hash12:02 PM
Smart Meter networks are pretty huge, but look like this in a very basic view

James Murphy12:02 PM
Hey Hash, assuming I have no testing hardware whatsoever what would it take for me to get in to this game?

I noticed so many routers on light poles today on my morning walk. Never really saw them before for some reason.

Nicolas Tremblay12:03 PM
2 finger and a plug?

james12:03 PM
That is not necessarily a correct view.

Hash12:03 PM
@James Murphy Could start with a RTL-SDR, learn the basics of RF and SDR and you're well on your way for $30

Wim Ton joined the room.12:03 PM

james12:03 PM
Not all smart meters are mesh

John joined the room.12:03 PM

James Murphy12:03 PM
RTL-SDR ??

Hash12:04 PM
@james Indeed, I am specifically looking at Landis+Gyr

Hash12:04 PM
@Dan Maloney You'll see them all over the place now!


https://www.rtl-sdr.com/about-rtl-sdr/

rtl-sdr.com

About RTL-SDR

What is RTL-SDR? RTL-SDR is a very cheap ~$25 USB dongle that can be used as a computer based radio scanner for receiving live radio signals in your area (no internet required). Depending on the particular model it could receive frequencies from 500 kHz up to 1.75 GHz.

Read this on rtl-sdr.com

james12:04 PM
Yes, The comm module on a Landis Gyr can be replaced.

nicklapolis joined the room.12:04 PM

Hash12:04 PM

baldrick (NE2Z)12:04 PM
As @Hash was saying ..

Hash12:04 PM
This a view of the boards I am analyzing

dbcorbin12:05 PM
how can you tell what tech is inside the smart meter on my house?

james12:05 PM
Ususally depends on the Utility for the tech.

weberzach joined the room.12:05 PM

James Murphy12:05 PM
Thank's Hash...

Hash12:05 PM
@dbcorbin Take a pic of it and post it here, if it has a FCCID then you got something worth analyzing

Those two long zig-zaggy chains of resistors are curious...

loop1712:06 PM
https://fccid.io/ is better than the offficial FCC site for looking that up

andrellobbello joined the room.12:06 PM

James Murphy12:06 PM
FCCID ??

loop1712:06 PM
yes, the identifier for the FCC

John12:06 PM
Some utilities still use power-line carrier systems.

Hash12:06 PM
Yea, I am working on reverse engineering the layout of the PCB as well....

james12:06 PM
https://sensus.com/communication-networks/sensus-technologies/flexnet-north-america/ list an alternative that is not mesh.

Hash12:06 PM

weberzach12:06 PM
@James Murphy note the FCC id in his photo.

mikethibodeaux53 joined the room.12:06 PM

james12:06 PM
It is what is on my house.

felix106312:07 PM
is there no audio sound with these chats?

James Murphy12:07 PM
Thank's Weberzach

No, text only

felix106312:07 PM
ok. thanks.

weberzach12:07 PM
@Hash , are you interested in their ability to communicate, or measure usage accurately?

Hash12:07 PM
@felix1063 Like oldschool IRC Hacking days!

Wim Ton12:07 PM
The zigzag resistors are used because a single resistor is not specified for the full mains voltage

James Murphy12:08 PM
Are they reporting Brown-Outs?

Hash12:08 PM
@weberzach I am interested in the mesh network, how they route messages, what messages get sent etc

Hash12:08 PM
@James Murphy They report power outages for sure, likely line conditions and brownouts too but not sure what data they would send for that

dbcorbin12:08 PM
Mine is a PGE FCC-id: OWS-NIC514 Silver Spring networks

felix106312:09 PM
Hod do they measure correct power factor?

richard12:09 PM
I've been using one of these with my meter (but it requires you do a setup with the power company): https://www.rainforestautomation.com/rfa-z105-2-emu-2-2/

weberzach12:09 PM
@hash is there any easy way to monitor with an RTL my own home's usage? Or is the best bet still the "IR" sensors?

James Murphy12:09 PM
Thanks Hash...

Hash12:09 PM
@felix1063 They use a chip made by Teridian (Maxim now) to do that

baldrick (NE2Z)12:09 PM
@hash what is the protocol format?

baldrick (NE2Z)12:09 PM
and is it LoRa on 900 Mhz ?

Hash12:10 PM
@richard I have one of those as well but in Dallas they killed that functionality recently

Scott H12:10 PM
Mine is Open Wave, FCC ID: SK9ACT1

james12:10 PM
Depending on the Utility. The communications can be encrypted.

Hash12:10 PM
@baldrick (NE2Z) More info on the protocol here. https://wiki.recessim.com/view/Landis%2BGyr_GridStream_Protocol

baldrick (NE2Z)12:10 PM
cheers

Denver12:10 PM
Have you checked out rtlamr?

Hash12:11 PM
9600 baud with start/stop bits

Dale joined the room.12:11 PM

Hash12:11 PM
Yea, it doesn't work for these meters unfortunately

farmboy12:11 PM
RTLAMR doesn't apply to smart meters.

Denver12:11 PM
Ah, thanks

Hash12:12 PM
Landis+Gyr engineers wrote a paper about how their routing protocol works that was very interesting, here's an excerpt

Hash12:12 PM

loop1712:12 PM
link to the paper?

Hash12:12 PM

Hash12:13 PM
I had to pay for it on IEEE...

Hash12:13 PM
but might be available somewhere with some googling

Hash12:13 PM
The idea of a geographic routing protocol was very interesting


https://pdfs.semanticscholar.org/5ab2/6a0c8722d29e3780ac77310f07388a674d43.pdf

Semanticscholar
Read this on Semanticscholar

The PDF looks like a presentation based on the paper, could be useful

Maxwell Faraday12:14 PM
PG&E has partnered with NTS to provide real-time consumption data. It appears that they validate devices that are able to become nodes on PG&E's Zigbee network. These devices may provide some insight. https://www.nts.com/services/certification/pge/han-devices/

James Murphy12:14 PM
Thank's Dan!

Hash12:15 PM
Yea, that's a solid presentation

Hash12:15 PM

Hash12:15 PM
The paper and testing was done in Dallas where I live!

farmboy12:16 PM
there was a lot of mesh research going on.... back in 2000s. this looks like a university paper.

farmboy12:16 PM
what's the date on it the IEEE doc from L&G? i wonder if that just ended up becoming the "standards" for field area network routing... . published in Wi-Sun/ RPL / 802.15.4g ?

richard12:16 PM
I see some of the meters use the zigbee standard, would getting a USB zigbee adapter to sniff zigbee packets be helpful?, something like this: https://www.microchip.com/DevelopmentTools/ProductDetails/AC182015-1

Hash12:16 PM
@farmboy It's from the same IEEE paper

farmboy12:16 PM
zigbee isn't the network hash is decoding.

Hash12:17 PM
@richard These use Zigbee for a local home area network, and 900MHz proprietary mesh for comms back to power company

James Murphy12:17 PM
Thank's Richard!

Hash12:17 PM

richard12:17 PM
ahh ok, thanks

farmboy12:17 PM
@Hash what do you suppose is the next step decoding this L&G fan?

Hash12:17 PM
You can see under the RF cans on this pic, left side zigbee, right CC1020 for mesh

james12:18 PM
There is usually a daughter board for the actual comm.

farmboy12:18 PM
cc1020 is publically documented - so.... if there is a static key in there... you could get it sniffing the spi bus when it boots

Todd Christell12:18 PM
@Hash what software are you using to explore/decode the protocol? GNU Radio?

So then what is the Zigbee network talking to? Stuff inside the customer property?

farmboy12:18 PM
(assumming the key is in the host processor)

Hash12:18 PM
@james On these ones it's all one PCB, I have some others that are split

Bernard joined the room.12:18 PM

Hash12:18 PM
@farmboy Decode the power data, so far I don't have it yet

Wim Ton12:19 PM
@farmboy The keys are different for each meter.

Hash12:19 PM
@Todd Christell Yes, custom block I wrote to decode L+G and Frequency hopping utilities by Sandia Labs

Bharbour12:19 PM
Where do you get electric meters from?

Hash12:19 PM
@Dan Maloney Correct, zigbee to consumer

Hash12:20 PM
@Bharbour eBay!!

SO maybe one of those little dongles that customers can use to view their usage, etc?

Hash12:20 PM

https://www.ebay.com/sch/i.html?_from=R40&_trksid=p2380057.m570.l1313&_nkw=landis%2Bgyr&_sacat=0

Ebay

1,409 results for landis+gyr

Read this on Ebay

andrellobbello12:20 PM
@Hash do you know of any cheap enough hardware to play with zigbee? i tried the ApiMote but stopped working after a while. I have a hackrf too but it can't do duplex tx/rx. didn't find any reasonable alternatives... :/

Bernard12:20 PM
I used to have one of these dongle and SCE stopped supporting them

Hash12:21 PM
@andrellobbello I'd say get a board dedicated to ZigBee, I haven't worked with it though

Bernard12:21 PM
SCE: Southern California Edison

Hash12:21 PM
@Bernard Same here in Dallas

farmboy12:21 PM
yes, SCE gave up on SEP 1.x (zigbee). they shut down their home energy portal.

andrellobbello12:21 PM
Yeah the ApiMote was supposed to but it let me down haha thanks tho! :)

farmboy12:21 PM
HAN (home area network) ZigBee / consumer <- Meter -> FAN (field area network) / utility

dolsongte12:21 PM
Is anyone addressing forensic analytics to identify the hack after it happens?

Todd Christell12:22 PM
If I'm doing things correctly it appears to be a "chirp." I know that the water and gas communicate with the electric meter and that is higher power signal to an intermediate node so not sure which I'm seeing.

Hash12:22 PM
@dolsongte What do you mean?

Hash12:22 PM
@Todd Christell Your water/gas/power meters all made by same manufacturer?

farmboy12:23 PM
very few utilities cover gas, water, and electric.

farmboy12:23 PM
so all the meters are usually different.

farmboy12:23 PM
SCE is maybe the exception there.

Hash12:23 PM
That's how it is out here

Hash12:23 PM
all different

Todd Christell12:23 PM
@Hash Yes, they bought it as a package.

dolsongte12:23 PM
Database analytics that the utility may use to identify when and where the meter was hacked

Hash12:23 PM
Good for them, probably more efficient that way

felix106312:23 PM
what are some of the common rf frequencies used for communications of the network?

farmboy12:23 PM
you guys are going to love the NEXT generaiton smart meter. have you heard about it?

farmboy12:24 PM
400, 900, and 2.4

Wim Ton12:24 PM
And the meters have tamper switch

Hash12:24 PM
@felix1063 902-928MHz is what these use

farmboy12:24 PM
https://developer.itron.com/content/distributed-intelligence-introduction

Linux and WiFi. hackers DREAM

Hash12:25 PM
@farmboy That's going to be fun

Hash12:25 PM
@dolsongte I am sure at the head end system they have ways of detecting suspicious activity and flagging it

Dale12:25 PM
My utility in MI, DTE, uses itrons, and customer can also get a Powerley 'energy bridge' from zigbee->wifi/eth->cloud->phone app, and it even has an mqtt server on it that's open to subscribe to on local network. The meter configuration for what tariff or rate you have is programmed in by a technician using the IR interface, or they also can change the configuration remotely over the mesh network. Have you noticed different sets of messages on the mesh network depending on what tariff the customer has? ie: fixed flat rate, time of use, demand rate, etc.

farmboy12:25 PM
nothing like a stranded linux distro on the side of your house! connected with wifi and bot-net ready :)

felix106312:26 PM
Thanks. Do the meters transmit in a set interval and if so, what is that interval?

james12:26 PM
Interval is configurable by utility

Hash12:26 PM
@Dale I haven't noticed traffic differences there that I could discern, but it's a good idea of what to search for!

Wim Ton12:26 PM
Meters tend not to use Linux, too resource hungry

Hash12:26 PM
@felix1063 Mine transmit once a minute normally and power data every 15 mins

Hash12:27 PM
Lots of other traffic as well

farmboy12:27 PM
there's some old meters that only transmit once a month! (gas)

Hash12:27 PM
Uptime

james12:27 PM
That sounds like it got wiped to a default rate

Dale12:27 PM
Yeah, people worry about that with the energy bridge, it is linux. @farmboy I worry more about the utility screwing things up than hackers out here.

Hash12:27 PM
Gas meters I looked at were VERY interesting... Battery powered FOR 10 YEARS!

james12:27 PM
transmit is usually not that often to reduce traffic

SimonAllen12:27 PM
What worries me is ransomware. Bad guy targets the utility company. Accesses their system. Turns off everyone's meter then encrypts their system and sits back waiting for a ransom to be paid before giving them the key. Meanwhile, the end-user sits in the dark.

farmboy12:28 PM
@Dale that's a good point. utlities are kinda only good at... uhhh... collecting bills and turning off your power

Hash12:28 PM
@james Nope, all the live meters around here transmit once a minute

Bernard12:28 PM
Gas meter; 10 year battery, this is why they cannot transmit very often

Hash12:28 PM
TONS of traffic to snoop

Dale12:28 PM
@SimonAllen yeah

james12:28 PM
Not all meters have remote disconnect.

dolsongte12:28 PM
@hash, not necessarily can the head end systems detect it. Depends on the utility's scada/ems systems.

Wim Ton12:28 PM
Firmware update are digitally signed on modern meters

james12:28 PM
I know of water meters with 20 year batteries.

So yeah, that brings up a good point -- what about powering water meters? Battery too I'd imagine

Hash12:28 PM
@dolsongte That's true

Todd Christell12:29 PM
Yes, both out gas an water are battery powered which is why they communicate locally with our power meter which has the "available" power for a more powerful radio. They had to replace all of the water pit covers with plastic so the RF could get out :)

Hash12:29 PM
Never looked at water meters but probably...would be cool to power them based on water flow spinning some kind of generator

farmboy12:29 PM
^ i don't think you'll get the firmware of the meter.... but you can sure as heck get the data (before it goes over rf)... via the spi bus between the meter's micro and the radio micro.

Hash12:29 PM
@farmboy I already got the firmware off the meter :)

farmboy12:29 PM
awesome!

farmboy12:30 PM
anything good in there?!

Hash12:30 PM
Also the bootloader the M15C processor uses

Hash12:30 PM
M16C

james12:30 PM
Water flow is not a reliable power source. The meters degrade over time and need to be replaced every 20 years

Hash12:30 PM

Hash12:30 PM
Working on disassembling it now

farmboy12:30 PM

Discussions