New Year Project - Power Quality Logger.

[Crossy]

Here it is, what you've all been waiting for, the latest project to emerge from Crossy Labs.

 

A simple data logger to measure your power supply voltage and current, do a bit of maths then store the date, time, voltage, current, power and possibly a few more parameters (probably power-factor) onto an SD card for you to mangle later with Excel or a similar spreadsheet.

 

Unlike the pond controller this one may actually be useful for other members.

 

Many of us suffer from "interesting" power supply issues and often (always) have difficulty convincing MEA/PEA that there is a problem because (invariably) there's nothing going on when they call.

 

With this unit you will (should) have a record of what's been going on whilst they weren't looking.

 

Do note that this unit isn't going to compete with a "proper" power quality logger, they cost many thousands of dollars, but we should be able to isolate common issues. Well one can hope.

 

====================

 

Here we go with an initial shopping list.

 

I have purchased (prices are Yankee Dollars):-

 

An Arduino Uno clone ($3.20) https://www.aliexpress.com/item/Free-shipping-UNO-R3-MEGA328P-with-usb-cable-for-Arduino-Compatible-Dropshipping/32241225914.html

 

 

A handy-dandy bolt on card with a real time clock (with battery backup) and an SD card slot ($2.05) https://www.aliexpress.com/item/XD-05-Data-Logger-for-Arduino-Data-Logging-Shield-Free-Shipping/1467034213.html

 

A 20 character by 4 line LCD, so we can see what's going on in real time ($4.40) it will work fine without this, but it's a valuable debugging tool https://www.aliexpress.com/item/Special-promotions-LCD-module-Blue-screen-IIC-I2C-2004-5V-LCD-for-arduino-blue-screen/777032831.html

 

 

=======================

 

Coming soon, some code to hang the bits together and some hardware to actually measure voltage and current of your mains (safely of course).

 

[Forkinhades]

Looks great. About time I got my arduino out, and having a play.

Sent from my SM-N950F using Thailand Forum - Thaivisa mobile app

[Crossy]

A small update.

 

This is the datalogging board I bought.

 

The LCD connects to the I2C bus via A4 and A5 like on the pond controller. Currently no input hardware for volts and amps (coming soon).

 

 

And here's the initial sketch I have produced Datalogging_V1.4.ino as with the pond controller you can open with any text editor which understands Unix formatted text if you don't have the Arduino software.

 

Currently the software:-

• Reads the date and time from the real-time clock.
• Checks an SD card is inserted and initialises it.
• Writes the date and time to the LCD.
• Creates a data file on the SD card with the filename based on (month day hour minute).csv  so Excel will open it directly.
• Loops round writing data records in the file "date, time, loop count, random number". The random number is a debugging tool, the loop count is so we can distinguish between records which have the same time stamp.
• Writes 250 of these records before closing the file and creating a new one (will increase the count later, it's there to stop us making a file too big for Excel).

All good fun.

 

Measurement hardware and measurements coming soon

 

[Crossy]

Here we go with a small update.

 

The electronics, as you can see it's not exactly complex. I built it on a scrap of strip-board, didn't even need to cut any tracks.

 

 

The sketch that drives it Datalogging_V1.5.ino

 

And it working monitoring a test load (kettle)

 

 

Here's the raw data from the kettle boiling test PowerMonitor.xlsx I've added header rows and a simple graph of voltage for interest.

 

The system makes about 2.7 measurements per second, not going to catch surges but it should be good enough to go and annoy PEA if you have chronic supply issues.

 

I tweeked the calibration by adjusting the "fiddle factors" so it reads the same as my Kill-A-Watt meter, not perfect but good enough. At this point I noticed that the calibration factor for current was "somewhat" different from what I expected. This turned out to be because the current transformers I bought are 1:5000 not 1:2000, I'll be adjusting the load resistor accordingly once I box it all up.

 

Of course, like all good ideas it's been done before https://learn.openenergymonitor.org/electricity-monitoring/ac-power-theory/introduction so I filched the libraries to do the RMS sums from there, no point re-inventing the wheel. I will be updating to keep a running total of energy usage (in FLASH so it doesn't get reset by a restart).

 

More coming when it gets installed on our supply so we can see what's going on when I'm not looking

 

 

 

[Crossy]

The beast is sitting in my workshop monitoring the supply to the irrigation pumps and workshop. It's still not in a box so is not permitted in the main house yet.

 

Some (possibly) interesting observations.

 

Firstly it generates a LOT of raw data, with the file limit set to 60,000 samples (so Excel can load it) each file covers about 6 hours and is around 3.5MB. Four files per 24 hours = 14MB, the 2GB SD card I'm using is in no danger of filling up (just under 5 months of data will fit).

 

I also suspect some interference between the voltage and current sensors, possibly because they are sharing the Vref (2.5V) source. Time to add buffers to the reference voltage methinks.

 

See the plot. This is our 2HP irrigation pump starting, I believe the current spike (green line) and reduced voltage (red line) during the run, I don't believe the 340V voltage spike (I would expect the voltage to dip during the startup surge). I really (really) hope it's a measurement error!! 

 

 

On the whole it's going fairly well and providing suitable brain exercise. And I have remembered how to solder properly 

[Forkinhades]

Indeed, making my grey matter hurt already.

Sent from my SM-N950F using Thailand Forum - Thaivisa mobile app

[Crossy]

On 2/13/2018 at 12:45 AM, Crossy said:

I really (really) hope it's a measurement error!

It was

 

I separated the Vref supplies for Voltage and current and got something much more sensible.

 

One can see the 15A (about 3x run current) start surge of the 2HP pump (single phase induction motor) and the associated volt drop.

 

The 6A run current causes about 3.5V drop equating to a supply impedance of around 0.58 ohms which matches reasonably well with my measurements made when specifying our AVR (there's an extra 15m of 4mm² from where those tests were made), see the pinned thread.

 

There's still a good deal of jitter on the voltage measurement but (oddly) not on current. I'd like to apply some smoothing but the start surge only lasts for 2 samples, don't want to disguise this type of thing.

 

 

Investigation continues.

 

Like British Rail - "We're getting there".

 

[Crossy]

Applying my jaded brain to the voltage "jitter" issue I suspect it's related to the Arduino Uno's 10 bit ADC (Analogue to Digital Converter).

 

We are sampling voltage and current a number of times per cycle to make the RMS calculations, nothing is done in hardware except scaling the input. Of course this means that the mains voltage to be measured goes from -325V to + 325V (230V RMS). That's 650V being chopped up into 1024 (10 bit) measurement steps = 0.6V per step (of course this is scaled and offset to fit the 0-5V input range of the Arduino).

 

I may invest in an Arduino Due, it has 12bit ADCs so we get 4096 steps = 0.16V per step. The libraries I have will work with the Due in 12 bit.

 

Unfortunately I need to same up my pennies as the Due is $12 

https://www.aliexpress.com/item/Due-R3-Board-DUE-CH340-for-arduino-ATSAM3X8E-ARM-Main-Control-Board-with-1-meter-USB/32847070719.html

 

 

[Crossy]

Update time, since we had Friday off (company has Chinese connections so CNY is a holiday) I got on with the task in hand

 

The beast is now in a box, so is allowed in the lounge with the electrical panel. Madam hasn't complained about the wiring, yet! The outlet is straight off the incoming supply (via a 6A MCB) so power and voltage monitoring via one wire.

 

 

The innards.

 

 

And the outards.

 

 

The current transformer load resistors live in the yellow heatshrink "power bulges".

 

An important note about current transformers. Never, ever, leave them with the secondary open-circuit when on an energized supply. Even the baby ones we use on this job can generate hundreds of volts, not going to kill you but you may get a rude awakening.

 

My CTs gave me a headache, the blurb on AliExpress describes them as "100A/20mA" making them 1:5000. Having set up and calibrated the beastie with 85 Ohm load resistors all seemed well. Until I loaded up past 24A or so, at 37A load we were still reading 24A, oops.

 

A quick look at the output voltage said 2.4V, way over the 1.5V rms maximum for the Arduino analog input. 2.4V across 85 Ohms = 28mA. 37A / 28mA gives a ratio of  1: 1300 so the transformers are probably 1:1500 (maybe 2000). Grrrrr.

 

Re-calculating the load resistor put things at about 25 Ohms, I bunged 39 Ohms in parallel with the existing 85 Ohms (for 26.73 Ohms, close enough to calibrate out) hence the two bulges on the CT wire.

 

 

[Crossy]

Some test results, now I've fettled the current reading.

 

 

Voltage is the blue trace (left side axis) current is the orange trace (right side axis). The big amps were me testing with our "carefully calibrated variable load" (OK the big water heaters). Got up near 60A (our poor 15/45 meter was crying in pain) with broad agreement to the supply meters.

 

So now we do the 24 hour monitoring test Let's see what our power really does when nobody is looking

 

[Crossy]

The electronics now looks like this

 

 

And the software like this Datalogging_V1.9.ino

 

 

[Crossy]

After cooking for 24 hours we have 13MB of CSV data comprising 232,207 samples (did I say it generates a LOT of data?).

 

Loaded into Excel 2013 (does up to 1 million rows) and plotted it looks like this:-

 

 

All a bit overwhelming, split into 6 hour chunks it's more manageable.

 

 

 

 

All still a bit complex, but one can see known events such as our morning showers at 06.28 (me) and 06.37 (Wifey) and the same in the evening 20.15 (me) and 20.21 (Wifey). If nothing else it serves to show Wifey spends 3 times as long showering in the evening as I do (not sure if this is good or not). You can also see the initial run of the bedroom A/C after the showers.

 

We also see garden watering going on at 16.31 (3HP pump) then the 2HP khlong pump starts at 16.37 with watering stopping at 17.11 and the khlong pump stopping at 17.33 having topped off the tank.

 

Looking at the 00.00-06.00 graph you can see the fridge and freezer cycling and me turning on the A/C at 04.07 coz I was hot (there was a reason for being hot ).

 

EDIT You can also see the house water pump starting at the end of the A/C run after a bathroom visit. Cripes, this really is revealing waaaaay too much information.

 

Also, the 18.00-24.00 plot shows a definite fall off in supply voltage over the evening, culminating with our taking the evening showers at absolutely the worst time and pulling the supply down towards 190V.

 

What does all this tell us, apart from revealing the timing of our bedroom activities (after 14 years of marriage I'm used to being awakened at all hours and being expected to perform) to all Thaivisa members? Not sure yet.

 

 

 

 

 

[Crossy]

Another day, another plot.

 

Supply looks a lot cleaner on a weekday, I was thinking the spikes on the current were a measurement aberration but there are a lot fewer than at the weekend.

 

All day.

And 6 hour chunks.

 

 

[Crossy]

OK, this is probably the "final" software update.

 

Correctly displays month, day, hour, minute, second with leading zero if <10.

Does the same when creating the .csv files so they sort properly.

 

Datalogging_V2.0.ino

 

Also a macro-enabled Excel 2013 sheet with a VB script that imports the .csv files (oiked off the net).

 

Power Monitor.xlsm

 

1. Select cell A1 of Sheet 1
2. Hit alt F11 to run VB
3. Select module "Import_CSV"
4. Hit F5 to run
5. Follow the prompts.

 

The charts on the second sheet should now be filled correctly.

[Crossy]

Here you go, sorry I didn't include the link earlier.

 

https://www.extendoffice.com/documents/excel/3388-excel-import-multiple-text-csv-xml-files.html

 

[Crossy]

One "final" update, this simply forces a new data file to be generated at midnight making life easier when you want to import into Excel.

 

Datalogging_V2.0.ino

 

Now we wait for something interesting to happen.

 

Next project time methinks.

 

[Daffy D]

Crossy, I envy you the time and resources for your projects.

 

Back in the old days on a planet far far away I used to have similar hobby. On this far far away planet it was easy to get parts and modules from places like RS and Maplin. Unfortunately I don't now have the time or resources or the money to indulge my hobby anymore   (wife kids and all that goes with it)

 

Next project? How about your mates diesel jenny cut off switch?

 

Or would that be too easy for you?

[Crossy]

9 minutes ago, Daffy D said:

Next project? How about your mates diesel jenny cut off switch?

 

I was actually thinking of that, maybe "universal" like this beastie https://www.gencontrol.co.uk/auto-start-for-hyundai-genset.html which can simply replace the keyswitch of a genset with electric start/stop and an automatic choke if it's petrol.

 

We can put something together with more functionality for a lot less than the 102 Squid that unit costs.

 

Time I have in the evenings and weekends, no kids and I really don't watch much TV. Wifey looks after her plants and I do "mysterious things" involving a computer.

 

Madam did make a comment that our electrical panel looks like a swarm of fireflies, there are "a few" LED indicators and LCD displays.

 

[Forkinhades]

Love it. 2 kids here, but not working when I'm here in LOS. Don't watch tv either.

Sent from my SM-N950F using Thailand Forum - Thaivisa mobile app

[Forkinhades]

...without hijacking the thread. Xy was wondering if you know where I could get a start-up electronics kit for my boy (me). [emoji4]

 

Sent from my SM-N950F using Thailand Forum - Thaivisa mobile app

 

 

 

 

[Daffy D]

43 minutes ago, Crossy said:

I was actually thinking of that, maybe "universal" like this beastie https://www.gencontrol.co.uk/auto-start-for-hyundai-genset.html which can simply replace the keyswitch of a genset with electric start/stop and an automatic choke if it's petrol.

That seems like a project worthy of your talents  

[Crossy]

1 hour ago, Forkinhades said:

...without hijacking the thread. Xy was wondering if you know where I could get a start-up electronics kit for my boy (me).

Not something I've seen here, but I've not looked either. 

 

How about one of the Arduino based hobby robots, simple electronics to interface and it does something interesting?

[johng]

@Crossy   could you give links for purchase of the transformers ?

 

also how/what is the current transformer  A-E side connected  to....

errm  never mind I think I worked it out.   ???? 

https://learn.openenergymonitor.org/electricity-monitoring/ct-sensors/interface-with-arduino

 

 

In a new Local 7/11

 

[Crossy]

6 minutes ago, johng said:

could you give links for purchase of the transformers ?

These are the CTs I used, read the thread to see my saga with the burden resistor. Also don't use the same voltage divider for I and V unless you use op-amp buffers to prevent interaction. 

 

https://www.aliexpress.com/item/5PC-Precision-AC-current-transformer-coil-PZCT-100A-20mA/32609298932.html

 

Voltage transformers came from NPE in Zeer, the smallest mains/12V trannies I could find, tiny cost.

 

[johng]

Well one small setback from Lazada

...why on earth would a transformer be a prohibited item !!! ????

[Crossy]

1 hour ago, johng said:

Well one small setback from Lazada

...why on earth would a transformer be a prohibited item !!! ????

It shouldn't be of course.

 

But I've had the occasional item cancelled due to "customs issues", one was a pack of breadboard jumper wires.

 

I'll bet it was simply they had no stock / couldn't get any.

 

I'd just order from another supplier.

 

[johng]

Eventually after 2 more goes I gave up and ordered this

https://www.lazada.co.th/products/watt-meter-4-in-1-voltage-current-power-watt-energy-electricity-meter-with-coil-ct-pzem-061-ac-80-260v-0-100a-multi-function-lcd-digital-display-btc-eth-miner-i227787961-s348264618.html

 

which comes with a current transformer as part of the package !

 

Thought about hacking it to talk wirelessly via an ESP826 wifi  board to my MQTT server (turn it to a smart meter)

https://www.domoticz.com/forum/viewtopic.php?f=38&t=7325&sid=4a9e05f550ffdd3afedfcc07ff79725e&start=60

and

https://gizmosnack.blogspot.com/2014/10/power-plug-energy-meter-hack.html

but  its using a Vango V9811A chip that nobody has published any successful method  and I can't work it out + the soldering pads are tiny and my hands shaky.    so its just a dumb energy monitor ( for now)

 

Yesterday I wired it all up..no smoke no flash bang ????

 

Taadaaa !!!

 

[sometimewoodworker]

1 hour ago, johng said:

Eventually after 2 more goes I gave up and ordered this

https://www.lazada.co.th/products/watt-meter-4-in-1-voltage-current-power-watt-energy-electricity-meter-with-coil-ct-pzem-061-ac-80-260v-0-100a-multi-function-lcd-digital-display-btc-eth-miner-i227787961-s348264618.html

I am confused. The linked meter has only 4 items 

voltage, current, power, energy.

 

though the one you pictured also has 

Frequency and power factor (the last will be very useful if we start getting charged for apparent power, think smart meters).

 

1 hour ago, johng said:

which comes with a current transformer as part of the package !

Wouldn't that usually be called a current sensor? 

 

All the meters that I've seen, have those included. 

[Crossy]

20 minutes ago, sometimewoodworker said:

Wouldn't that usually be called a current sensor? 

All the meters that I've seen, have those included. 

 

If it's the type with a toroid and coil I'd call it a "current transformer", if it's the alternative Hall-effect (electronic) type I'd call it a "current sensor".

 

In reality the terms are used interchangeably.

 

I'm more confused as to why johng wasn't able to buy the beasts successfully.

 

[johng]

30 minutes ago, sometimewoodworker said:

I am confused. The linked meter has only 4 items 

voltage, current, power, energy.

Sorry yes wrong link

https://www.lazada.co.th/products/watt-meter-4-in-1-voltage-current-power-watt-energy-electricity-meter-with-coil-ct-pzem-061-ac-80-260v-0-100a-multi-function-lcd-digital-display-btc-eth-miner-i311394406-s743830735.html?

is the 6 in 1  version