I LIVE IN THE WILD, THIS LOOKS WILD.
A BEAUTY LIKE ME
Surprises at the Edge - The SONY Spresense
A Needle in a haystack
The approach I took for this project is based 100% on state of the art Tech. Deploying Tech on the very #Edge (jungles, forests, national parks) is quite demanding and an important part of the project is to select the right tools to deploy on such a tough environment. Tough is the right word, the challenges you find in this context go from weather conditions, having near zero available electric power to no standard communication facilities. To add more fuel to the fire, I decided to go the #AI at the #Edge way. #MachineVision #SoundRecognition #LoRa are key aspects for my project, running on batteries!
While exploring tech is always a good thing for mid term goals, you have to make decisions for the near term in order to take solid steps in the development of the endeavor. The IT marketplace is a tricky one, buzzwords can lead you to dead-ends. Today, words like #Edge, #AI, #IoT are used for everything; even more, they are used together. In my particular case, Edge doesn’t mean the edge of the network or a device connected to the electrical outlet consuming lots of amperes to perform some inference on images; IoT is also something I perceive a little bit different from the market consensus. I was looking for the right balance regarding size, computing power, energy consumption, integration capabilities, quality, connectivity (not necessarily Internet connectivity, as devices get smarter, they can be quite autonomous) and sustainability.
A huge mistake
In the quest of finding the perfect fit I explored many options, The habitual industry names, I even got in contact with a very promising French start-up company, my criteria was the following:
• Processing Power vs. Energy Consumption
• The ability to run a real time operating system (#RTOS)
• Size and expansion options
• AI, cameras, audio capabilities on batteries
• A well established name to make the choice as durable as possible.
• Solid enough to be more than a prototyping / PoC tool
• Cost, which is quite important when it comes to deploy hundreds of devices in not secure environments.
Each try left positive findings, but I can’t forgive myself for not paying attention to what SONY was doing in the field. One day, by accident, I knew!
Solutions from Japan!
This is Buddha's heart. It has six outer strangles, two inner ones. I guess it is Japanese and if you manage to understand the relationship between time, events, causes and effects, then you have a big part of your life solved. That’s what people say.
This is the #Sony #Spresense, it has six cores, two additional MCU dedicated to GNSS (Global Navigation Satellite System that supports GPS, GLONASS and some others, you will get localization data all over the world, ideally outdoors unless you add an external active/passive antenna) and sensing capabilities (I/O, Hi-resolution audio) and power management; If you manage to understand what it really is, then you have a lot of problems solved in your tech life. That's what I say. So, what is it exactly?
Not a Toy, Not an expensive Arduino UNO from SONY, Not a lightweight Raspberry PI
The associative human brain can be tricky, the Spresense comes in an packaging similar to the official #Arduino packaging, it has pins to plug things in, it is even compatible with the Arduino pin-out (you can use some popular shields) and it can be flashed through the Arduino IDE but it also has a micro SD card and runs some sort of UNIX just like the PIs. Well, forget about those similarities because this board falls into another category.
From a hardware point of view, the Spresense is all about the Sony CXS5602 Microcontroller which is a six core (Cortex M4F) low power consumption unit, plus the two cortex (M4 and M0+) for the GPS and I/O, plus its on-board hi-res audio capabilities chip (CXD5247) with the quality you would expect from Sony, this last chip also handles the power management. From a concept point of view, and this opinion is quite influenced by my project needs, the minimalist approach taken by SONY is a key point. The inclusion of features like GPS, A reasonable quality camera, up to 8 mics (4 if analog) and a micro SD card reader makes it a perfect fit for Intelligent Devices at the Edge, of course, running on batteries.
The “missing” parts are what I like the most, you won’t find HDMI ports, nor WiFi, nor Ethernet, nor BLE. Generic modules are great to explore things in general, but when a given objective is in mind for a fast prototype or a PoC, those not needed features become a problem. Just to illustrate the case, take a look at the green card in the picture, that’s a LoRaWAN card with an integrated WiFi module (not an useful combination), while you can remove the energy feed to the WiFi module, I still paid for the module and the worst thing is that the size of the card is larger than required, compromising the size of the prototype.
So, the minimalist approach is great to me, if you need connectivity options you have, and you will have more add-ons (I understand they are working with lots of hardware partners to deliver more add-ons, some official ones I hope such as the upcoming LTE expansion board.), Third parties are welcomed by Sony. If you are in the field of hardware manufacturing, there is a complete guide on how to create an add on for the Spresense. You never know who reads these things I post.
​
As a first conclusion, Besides the powerful hardware, small building blocks allow you to reuse the early stage research in the following steps of the process, by doing so, you are reducing the gaps between research – prototype – final product. That is time and money saving. That’s a clever choice.
​
To be or not to be.
The Spresense is Arduino friendly, you download the board manager, you accept the terms and conditions, you burn the bootloader for the first time and you are ready to go. If you don’t pay attention to the log, you will never notice that you are installing a real time operating system (the #NuttX) each time you flash your software, even the fancy led example.
You can do some fast and heavy things from the Arduino IDE, you can even burn code for each core and make use of many features on the Spresense, but... the Spresense has a native SDK that allows you to fully control your device.
With the same reasoning I used before, I would say that working with the NuttX RTOS and the native SDK is a good investment regarding the reuse you will be able to do from your research and prototyping (Specially because NuttX is a POSIX compliant operating system). It could be a little bit more demanding than the Arduino way but once you mastered it, you will be able to do whatever you want, be it a drone, a smart camera, a personal assistants, you name it. There is also a VSCode extension to mange your projects.
In my particular case having and RTOS is vital, for the present and for the near future. I won’t tell you what a RTOS is but I want to remark two important characteristics of them. An RTOS is great for dealing with asynchronous, sporadic and short lived tasks. This characteristic is useful to handle many impulses from the outer world that could be competing to access shared resources. Taking into account that there are six cores to play with, the amount of sensors you can use effectively is high.
The low consumption cores can be even more effective in the responsible use of energy beyond its building materials and other secrets if you help them to do what they have to do only when necessary.
Just as an example, if your are planning on performing image inference at the edge, it could be a good idea to have movement detectors to awake the heavy processing tasks at the right time. The same is true if you want to perform sound recognition.
The Spresense will do the job even better if you help it. As an example, just plug digital sound detectors which will rise a signal on a pin when a certain level of noise is reached, you set an interruption on that pin and awake your code to take a picture / record sounds and run an inference on them, and then, go back to sleep.
The second critical point about having an RTOS is the low latency to attend situations, in real RTOS, those times are close to deterministic (you have deadlines, priorities and effective context switching that warranty that). Today, several drone navigations systems relay on NuttX for that reason. The choice made by SONY was good, NuttX is also POSIX compliant which means your code will be reusable on other platforms.
The complete picture
However, the Spresense is just the tip of the iceberg. Another nice surprise from Japan: I discovered their #AI tools. It turns to be that Sony is doing a great job regarding AI (proven technology used in their communication networks, robots and other customer appliances). Open Source communities and freely available (and amazing) tools complete the picture. What is important in this context is their nnabla library (evolved since 2010 when Sony started the research on AI) and the SONY Neural Network Console, a Windows application (I am running it on a MacBook Pro with Parallels with zero issues) that will allow you to train and create Neural Networks by drag and drop!.
You can find more information about this in the following links:
https://dl.sony.com (The NNC Console)
https://nnabla.org (The Library)
AI is a black box for many people, with this tool you will be able to arrange data for training (numbers, images, sounds... ) and create a NN by dragging blocks and connecting them together... just like that, drag an input function, then connect it with activators or math formulas, a final selector, save, train, evaluate and export the model to the C runtime. After optimization, that model will run on the Spresense using the DNNR (Deep Neural Network Runtime), you can do that even from the Arduino Environment.
Neural Network development has a lot of trial and error, both on datasets and the networks themselves, one thing that caught my attention was a feature to find structures in an automatic way, so, you can let your machine alone performing this repetitive tries to find the best evaluations... Maybe it gives you another good surprise. It is worthy to mention though that the Spresense won’t be able to run everything you create with this tool due to the obvious memory capabilities constraints on the device.
Sound amazing, right? Well, I don’t know if SONY is still using its Make.Believe slogan but in this case, my first impressions were unreal.
A summary from a technical point of view:
The Spresense has a powerful Microcontroller, runs a POSIX compliant RTOS, offers you the change to have image and sound on the Edge and runs AI networks on batteries, a 2D graphic accelerator and it is completely documented with a surprising level of detail that will tell you what kind of tape you have to buy if you lose the one that comes with the package!. That’s a good summary of the present state. I am wondering the kind of add-ons SONY and its partners are about to deliver in the near future.
One thing I would like to see from them is moving from the SoC (System on Chip) model to the SoM (System on a Module), that would reduce the gaps between stages even more and it allow many people to create quality products without a heavy understanding on circuit design. That would really empower makers to go a step further, even at the current stage, I think this board will rise the maker average quality standard, which should be quite welcomed because I see lots of old components (cheap and expensive ones) being sold to an unaware market. You can start doing things for hobby but in reality you never know how it is going to end.
A note of advice, the early adoption of technology (The Spresense is quite new) could be great and it could be also demanding and even risky. There is some work to do with the NuttX support still, so you can expect some troubles that will make you dress like a hacker. But, in response of that I would like to say a few words about Sony People spirit.
A few words about SONY’s people.
To be in Argentina has some negative things, but sometimes, limitations can lead you to great findings. After I discovered the Spresense I was having troubles trying to buy one, so, I contacted them. From that moment, my experience with Sony was, again, unreal. A week after the contact, the Spresense was here (Thanks Ramona again and all the people involved in the decision making process). The support team is also great. My “what is going on here?” moments didn’t stop there. Argentina’s time zone and my work habits unveiled another surprise. At my 3AM, I dropped a message on LinkedIn to a Sony division I have no contact with, I was asking for some information translated to English. My request was intercepted minutes after by a good mannered Japanese Manager and a few days after the information was in my inbox (Thank you Hiroyuki-san).
​
This is what I call a good experience! I think they understand the value of people.
​
** This review was written in 2019, things kept great since then.
Stay Tuned, Don't miss a thing,
Subscribe to the site.
ABOUT BCB
Big Cat Brother is a network of intelligent connected devices built to mitigate the common death causes that are putting Jaguars at direct risk of extinction.
​
If you are a Biologist, an NGO, Keep in Touch
​
​