No comments

What time series database to use for IoT systems?

  By

While most of the tutorials online nowadays describe influxdb as a very good time series database we discovered that there are some serious considerations you should do before selecting a time series database for your next IoT project.

Here are some findings we did and questions we should have asked when we researched and selected time series databases for a new design of an IoT system:

  • To design a stable database engine suitable for production typically takes 10+ years of user feedback and experience from smaller scale testing and continuous improvements. Several of the “high ranking” databases has not been around for that long
  • mySQL and ORACLE have stable database engines by now, but they are not time series databases
  • Influx has rewritten their database engine several times over a period of few years. This may be a hint to that they are still struggling to converge on a stable database design
  • We found several very worrying statements in bug reports related to influx such as “please help, our production database is suddenly corrupt”, “the database has crashed, the backup is also corrupt. There must be something fundamentally wrong with storage”, “we have discontinued functions iusedthisextensivelyandthereisnowayaroundit() in the next release due to a complete rewrite of the database engine” etc.
  • Some databases have very inefficient compression algorithms. They will compress, but when trying to read from a compressed table, the penalty is pretty big and it goes too slow forcing you to use uncompressed tables
  • Some databases do not support writing to compressed tables (!). You have to decompress before writing. This takes a lot of time. And there is no API support to check if you try to write to a compressed table. You have to resort to parsing log messages before writing. Yes it is true.
  • Even if a database shows up high in a ranking, it may be because of it has the highest growth rate. That does not tell you much. If you look closer, you may discover that all the big guys already use more conservative and well proven time series database systems. That people reading tutorials download and install influxdb does not mean that Google, Facebook, Amazon etc uses it. It does not mean that it is the best time series database available.
  • Some databases claim to support continuous aggregates. However did you benchmark and test properly that the functions used for reading from the aggregates are stable, does not require 100% CPU, and does not crash your database engine? You may be in for a surprise.
  • Some time series databases does not have support for regular data types and regular SQL syntax. This means you will have to have TWO databases. One for configuration data and one for time series.
  • Some time series databases from the cloud vendors are extremely expensive as soon as you come up into production. Did you actually check the cost? If not you may be in for a BIG surprise.
  • Even plotting time series in a web browser may be too slow. Did you check if your time series database can decimate and deliver data to your front end in real time?

 

No comments

Hva eiendomsmeglerne ikke skjønner når de selger bolig

  By
  • Det er aldri bilder av garasjen. Husk at for mannfolk så er garasjen en vesentlig del av boligen. Hvor mange salg og budrunder går selgere glipp av fordi det aldri er bilder av garasjen på finn.no?
  • Det er ofte ikke bilder av hobbyrom og rom i kjeller. Samme begrgunnelse som over. Mannfolk er ofte mer opptatt av dette enn mikrosement på badet.
  • Å ikke ha med selger på visning er i mange tilfelle idiotisk. Særlig når megler står inne i et hjørne med hendene i lomma eller snuta nede i mobilen og ikke sier et kvekk mens potensiell kjøper lurer på masse som ikke megler kan svare på.
  • Skal meglerne få solgt en bolig så må meglerne følge opp potensielle kjøpere å snakke med dem. Dette må gjøres på telefon. Det må stilles spørsmål, det må informeres, selger må konsulteres. Dette gjøres i for liten grad.
  • Skal megler få solgt en bolig så må megler vite noe om boligen. Ikke bare referere til prospekt. Mange meglere aner ikke noe om boligene de selger nå til dags. Skyldes dette allt for mange oppdrag, for mye press, for høy lønn? Ikke vet jeg.
  • Å ikke la potensiell kjøper snakke direkte med selger. Tenk så mye arbeid og misforståelser som kunne vært unngått. Og tenk så mye positivt om boligen (som tross alt selger kjenner best) som kunne vært formidlet. Hva er meglerne redde for? Hadde en megler som dekket seg bak personvern. For noe tull. Hvem som eier en bolig er jo offentlig informasjon. Alle står fritt til å ringe hvem man vil. Dette har ikke noe med personvern å gjøre.
No comments

Glitch protection for tube amplfiers with HV PSUs

  By
 

OE5JFL has implemented the same idea for quenching the HV discharge current as I wrote about back in approx 2012. The idea is to short the anode PSU caps thru an inductor and a solid state device when an overcurrent event thru the tube occurs. The inductor prevents the current to raise to dangerous levels as the inductor tries to prevent the current dI/dt by building a magnetic field.

The above figure1 shows OE5JFLS thyristor circuit and 100-500uH inductor

Thyristors for voltages >1000 Volts are very expensive. For higher voltages you can use a cascade as shown above in Figure 2. I use four thyristors in series, switched by optocouplers, in my 3.5kV supply.

Figure 3 above shows a circuit for screen protection sreen in case of no anode voltage or screen overcurrent. Ig2>100mA will toggle the Flip-Flop via the optocoupler. The relay goes off and disconnects the screen. Voltage comes back after pushing the RESET key. Using the ordinary Z-diode stabilisation, overcurrent is limited to a harmless value during switching delay of the relay.

Overcurrent trip: High current in anode circuit caused by arcing or short circuit generates voltage drop across 10Ohm resistor. Flip-Flop toggles (Pin 3 low), switches off transformer and fires SCR to short HV capacitor.

For restart one has to switch off S1, the main power supply switch, and switch it on again after a few seconds. The series inductance for SCR protection is made from 10m of enamelled copper wire. Trip current is set by the 39k/8k2 voltage divider.

Before you test the circuit by producing a short, you should follow the step by step procedure, which is outlined as follows:

  1. Connect TP2 to ground, switch S1 on and short Pin 6 (CD4011) to ground. ===> Transformer must be off.
  2. S1 on, after plate voltage coming up, ground TP1 (transformer off), and then Pin 6 (CD4011). ===> SCR must fire now and ground high voltage.
  3. S1 on, ground Pin 6. ===> Transformer must be off and HV must be shorted.
  4. After successfully completing these tests, you should short the HV output with a thin wire to ground (0.12mm). This wire must not be melted when the HV-supply is shut down by the overcurrent protection.

PROCEED WITH CAUTION, THE AUTHOR IS NOT RESPONSIBLE FOR INJURY OR EQUIPMENT DAMAGE.

Credit to OE5JFL for the schematics and writeups. 

https://www.qsl.net/oe5jfl/flashover.htm

 

No comments

Solar wind event of 12.May.2021

  By

The 12.May.2021 solar event caused major openings on 10 meters and above. I did this screen capture of the solar wind intensity to save for reference. One noteable observation is that the Bz magnetic measurement went to negative 20 for a moment. That is pretty significant.

No comments

How to convert binary fractions to decimal ( fixed point arithmetics )

  By , , 

Example for n = 110.101

Step 1: Conversion of 110 to decimal
=> 1102 = (1*22) + (1*21) + (0*20)
=> 1102 = 4 + 2 + 0
=> 1102 = 6
So equivalent decimal of binary integral is 6.

Step 2: Conversion of .101 to decimal
=> 0.1012 = (1*1/2) + (0*1/22) + (1*1/23)
=> 0.1012 = 1*0.5 + 0*0.25 + 1*0.125
=> 0.1012 = 0.625
So equivalent decimal of binary fractional is 0.625

Step 3: Add result of step 1 and 2.
=> 6 + 0.625 = 6.625
Example for n = 4.47 k = 3

Step 1: Conversion of 4 to binary
1. 4/2 : Remainder = 0 : Quotient = 2
2. 2/2 : Remainder = 0 : Quotient = 1
3. 1/2 : Remainder = 1 : Quotient = 0

So equivalent binary of integral part of decimal is 100.

Step 2: Conversion of .47 to binary
1. 0.47 * 2 = 0.94, Integral part: 0
2. 0.94 * 2 = 1.88, Integral part: 1
3. 0.88 * 2 = 1.76, Integral part: 1

So equivalent binary of fractional part of decimal is .011

Step 3: Combined the result of step 1 and 2.

Final answer can be written as:
100 + .011 = 100.011
No comments

Arduino not visible on COM port. Cannot upload a sketch even if you have flashed a fresh booloader. Solution.

  By ,

If your Arduino does not show up on any USB COM ports even if you flashed a fresh bootloader, it may be because the sketch you flashed has messed up your USB serial settings on the Arduino. This sketch will prevent the Arduino showing up as a com port in Windows because the flashed sketch starts immediately upon reset and “blocks” the normal USB COM operation.

Solution:
1) re flash the bootloader with an ISP connected to the 6 pin header. I use a USBasp (cheap ISP clone).

2) Just so you KNOW the bootloader is installed. You can check that the bootloader is operational by plugging in the board thru the USB COM port (when you already have an open hardware manager windows in Windows).
You should see briefly a new COM device named “Arduino [name of board] bootloader”. This  COM device will quickly disappear again if you have a sketch messing up the serial settings.At least you know the bootloader should be OK.
3) Upload a completely empty sketch WITH THE ISP device. Important!:Do that upload with the ISP and NOT via the USB serial cable.

This is a completely empty sketch you can use:

void setup() {}
void loop() {}

I have fixed several dead Arduino boards that I suspected had hw problems this way. In fact the boards were OK. The problem was that a sketch already installed messed up the serial settings. The bootloader and hw was completely ok.

No comments

RTOS on Arduino

  By ,

FreeRTOS Task for Blink LED in Arduino UNOProfessional software developers of time critical code often use a real time operating system (RTOS) in their work. The reason is that the OS is taking care of timing and scheduling tasks with higher priority. Now you can easily use an RTOS on your Arduino as well! Read Jain Risbah’s article over at circuitdigest: https://circuitdigest.com/microcontroller-projects/arduino-freertos-tutorial1-creating-freertos-task-to-blink-led-in-arduino-uno