Software Installations: 2018

Monday, November 5, 2018

MacOS commands

MacOS commands

Task Manager

<command><option><esc>

Screen Shot:

<command><option><h><m> -> Minimize all windows
<command><shift><3> -> copy entire screen to an image on Desktop;
<command><shift><4> + <space> + click -> copy the select window to an image;
<command><shift><4> + define a custom area with the mouse -> copy the selected area to an image;
<command><shift><5>

Ifconfig -> Find your IP Address;
miniterm.py /dev/tty.usbserial-1420 115200 -> Connect to serial device;
<command><shift><g> -> "Go to folder" in Finder
sudo chflags nohidden(or hidden) directoryname -> directoryname that you want to see(or not) in Finder. (log out and back in)
defaults write com.apple.finder AppleShowAllFiles TRUE -> Showing all files in Finder. (log out and back in)
SSH:

Check if SSH is enabled:

sudo systemsetup -getremotelogin

Enable / Disable SSH server (remote access):

sudo systemsetup -setremotelogin on
sudo systemsetup -setremotelogin off

sudo nmap -sn -PE 192.168.1.0/24 -> Ping all 256 IP's of the class C network 192.168.1.0. Using /24 will search 256 address, /23 will search 512 address, /22 will search 1024. Options meaning: -sn (No port scan) -PE (ICMP ping echo).
ping -c 3 192.168.1.255 | grep 'bytes from' | awk '{ print $4 }' | sort | uniq -> Ping the broadcast address to return all devices on the network.
ping 192.168.1 | arp -a -> Ping all IP's and return their MAC address.
<command><option><esc> -> The basic “Force Quit Applications” window, which can be thought of as a simple task manager for Mac OS X.
<command><shift><Dot> -> Show hidden files on Mac via Finder. Repeat it to hide them again

Installing Tools on Macbook Air

macOS Mojave 10.14.1
Xcode 10.1

Goal:

Install development tools on MacbookAir (e.g. Arduino IDE, Fritzing, Python, MicroPython, Brew, Sourcetree Bitbucket Sync, NodeJS).

ToDo:

Status:

2018.11.05 - Working on it.

Install:

Installing Xcode:

Install Xcode from the App Store.
Open Xcode and install "complement"

Install Xcode’s separate Command Line Tools app:

Open Terminal:
xcode-select --install
OR Download it from https://developer.apple.com/download/more/ and install manually

Install Homebrew:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Enter your macbook user password

Configure Path:

nano ~/.bash_profile
Write the text below:
export PATH=/usr/local/bin:$PATH
Save the file using <control><o> and exit with <control><x>. Then activate the changes with the command below:
source ~/.bash_profile

Check that Homebrew was successfully installed:

brew doctor
This should be the output:
Your system is ready to brew.
brew update

Installing Python3:

Search for everything you can install, related to python:
brew search python3
python3 --version
Pip, Setuptools, and Wheel have also been installed.

Check if it is a 32 or 64 bits (python2 and python3)

python
import sys
is_64bits = sys.maxsize > 2**32

Python Virtual Environments:

Virtual environments enable you to have an isolated space on your computer for Python projects, ensuring that each of your projects can have its own set of dependencies that won’t disrupt any of your other projects. You can set up as many Python programming environments as you would like. Each environment is basically a directory or folder in your computer that has a few scripts in it to make it act as an environment.
Create a folder to python virtual environments:

mkdir ~/development
cd ~/development
mkdir python_envs

Installing Virtualenvwrapper:

pip3 install virtualenvwrapper
nano ~/.bash_profile

export WORKON_HOME=~/development/python_envs
MacOS:

export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python3
source /usr/local/bin/virtualenvwrapper.sh

Linux:

export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source ~/.local/bin/virtualenvwrapper.sh

Windows:

pip install virtualenvwrapper-win
In Windows, the default path for WORKON_HOME is %USERPROFILE%\Envs

Close the terminal and open it again, to reload the bash_profile.

Creating a new environment (this creates the my_project folder inside $WORKON_HOME folder):

mkvirtualenv -p python3 my_project

To use this environment, you need to activate it:

cd $WORKON_HOME/my_project
workon my_project

Within the virtual environment, you can use the command python instead of python3, and pip instead of pip3 if you would prefer.

pip3 install django

To get out of the virtual environment:

deactivate

To delete the virtual environment created:

rmvirtualenv my_project

Installing Sourcetree:

Download at https://www.sourcetreeapp.com/
Install and clone repositories at /Users/<username>/development

Installing NodeJS:

Download and install (double-click)

Bash History behavior:

Note that the history file is automatically truncated to the size of HISTSIZE (500 by default) if the user does not explicitly set the size differently. HISTFILESIZE will be automatically set with the same value of HISTSIZE, so don't need to worry about it.
There are two lists of commands, one in memory and another in file. The setting for HISTCONTROL acts over the memory list. When you type a command it will be appended to the list. If the command to add is equal to the last command in the list and the option ignoredups is set, the command will be discarded.
Add HISTSIZE and HISTCONTROL to ~/.bash_profile:

nano ~/.bash_profile

export HISTSIZE=70000
export HISTFILESIZE=70000
export HISTCONTROL=ignoredups:erasedups
shopt -s histappend
PROMPT_COMMAND="history -n; history -w; history -c; history -r; $PROMPT_COMMAND"

Close the terminal and open again to apply the changes.

Info:

References:

Thursday, August 23, 2018

Installing Ubuntu 18.04 32bits on Raspberry Pi 3

Ubuntu 18.04 32bits

Raspberry Pi 3
Python 2.7.15 / 3.6.5
NodeJS 8.10.0
MySQL 5.7.23

Goal:

Install Ubuntu on Raspberry Pi 3 with Python, Mosquitto, MySQL and NodeJS.

Status:

2018.09.15 - Devices don't work for more than 5 min. It dosen't responds to ping. As these devices (Temp/Hum and Door sensors) only read sensors and publish to Broker, maybe the problem is with this version of Broker or an incompatibility with Ubuntu 18.04 for Raspberry Pi 3, or a hardware issue. When we pointed out all devices to other version of Broker in another computer, there's no problem anymore. Action: Mount another machine, with Ubuntu Mate 16.04.2 running on Raspberry Pi 2 to be an IoT Server.
2018.08.23 - Ubuntu, Python, MySQL, Mosquitto and NodeJS are working.

Installing Ubuntu:

Download Ubuntu Image
Install DDRescue and unzip:

sudo apt-get install gddrescue xz-utils

unxz ubuntu-18.04-preinstalled-server-armhf+raspi3.img.xz

Identify MicroSD device name (/dev/sdX) and write to it:
lsblk
sudo ddrescue -D --force ubuntu-18.04-preinstalled-server-armhf+raspi3.img /dev/sdX
First Boot with the SD card: (User: ubuntu Password: ubuntu)
Connect with WiFi:

nano /etc/netplan/*.yaml (e.g. 50-cloud-init.yaml)

For wireless dhcp addressing:

network:

version: 2

renderer: networkd

wifis:

wlp3s0:

dhcp4: yes

access-points:

"network_ssid_name":

password: "**********"

Finishing the Wifi configuration:

sudo netplan --debug generate # make config files
sudo netplan apply # apply new configuration
reboot. # must reboot

Once connected to Internet, installing optional PPAs:
sudo add-apt-repository ppa:ubuntu-raspi2/ppa
sudo apt update
Upgrade new packages (optional):

sudo apot upgrade

Change server hostname (optional):

sudo hostnamectl set-hostname <HOSTNAME>
sudo nano /etc/cloud/cloud.cfg
Change preserve_hostname: false to:
preserve_hostname: true
sudo nano /etc/hosts
Add a new line after "127.0.0.1 localhost"
127.0.1.1 <HOSTNAME>

Installing Python:

sudo apt install python python3 python-minimal python-pip python-mysqldb
Install Paho MQTT Python Client:
pip install paho-mqtt
Install SQLAlchemy:
pip install SQLAlchemy==1.2.11

Install Mosquitto MQTT Server:

sudo apt update
sudo apt-get install mosquitto mosquitto-clients
Start Mosquitto Server:

mosquitto -d

Installing MySQL:

sudo apt install mysql-server
sudo mysql_secure_installation
To configure the root account to authenticate with a password, run the following "alter user" command. Then, run "flush privileges" which tells the server to reload the grant tables and put your new changes into effect:
sudo mysql
alter user 'root'@'localhost' identified with mysql_native_password by 'password';
flush privileges;
Creating a new user (iotdbuser) and granting privileges:
create user 'iotdbuser'@'%' IDENTIFIED BY 'user_password';
grant all privileges on iotdb.* to 'iotdbuser'@'localhost';
To connect to MySQL server remotely using MySQL client, edit the /etc/mysql/mysql.conf.d/mysqld.cnf configuration:
sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf
And change the "bind-address" from 127.0.0.1 to:
bind-address = 0.0.0.0
And grant access to root user from network:
grant all on *.* to root@'%' identified by 'root-password';
Once ready, reboot your MySQL server:
sudo service mysql restart

Installing NodeJS:

sudo apt install nodejs npm
Create a directory for the NodeJS projects:
mkdir ~/iot/wsk_cp
cd ~/iot/wsk_cp
npm init –y
sudo npm install mysql
sudo npm install express -g
sudo npm install request -g
sudo npm install ws -g

Testing:

Testing Mosquitto:
mosquitto_sub -t "outTopic"
mosquitto_pub -t "outTopic" -m "hello world!"
Testing MySQL:
mysql -V
sudo mysql
select user,authentication_string,plugin,host FROM mysql.user;
Connect to MySQL server over network:
mysql -u root -p '' -h <MYSQL_HOST_IP>

Troubleshooting:

References:

Sunday, July 29, 2018

Installing MicroPython on ESP8266 and ESP32

Windows 10
MacOS Mojave 10.14.1
MicroPython 1.9.4.8

Goals:

Use Python for Microcontrollers (MicroPython) on Wemos D1 mini (ESP8266) and ESP32.

Info:

MicroPython is a condensed, optimized code of Python 3 that has been loaded in hardware. This means rather than having to have an interpreter run on an operating system to execute the Python code, the MicroPython chip can run the Python code directly on the hardware. No operating system is needed. In fact, MicroPython has basic file I/O built in.
After a fresh install and boot the device configures itself as a WiFi access point (AP) that you can connect to. The ESSID is of the form MicroPython-xxxxxx where the x’s are replaced with part of the MAC address of your device (so will be the same every time, and most likely different for all ESP8266 chips). The password for the WiFi is micropythoN (case-sensitive). Its IP address will be 192.168.4.1 once you connect to its network.

Installing Micropython:

Install the latest EspTool (2.5.0+):
Windows: using Windows Terminal (cmd):

For using ESP32 on WIndows we need to install USB Driver:

Download the driver and follow the instructions here.

pip install esptool --upgrade
pip3 install adafruit-ampy --upgrade
set AMPY_PORT=COM?

Linux:

sudo pip install esptool --upgrade
pip3 install adafruit-ampy --upgrade
export AMPY_PORT=/dev/ttyUSB0

MacOS: using Terminal:

pip3 install esptool --upgrade
pip3 install adafruit-ampy --upgrade
export AMPY_PORT=/dev/tty.usbserial-1420
For using ESP32 on MacOS we need to install USB Driver:

Download Legacy MacVCP USB to Serial driver
Install SiLabsUSBDriverDisk.dmg
Restart MacOS
After restart, a new device will be detected (e.g: dev/tty.SLAB_USBtoUART)

Erase the flash (ESP8266 and ESP32):

Windows:

locate the latest esptool.py installed:

cd \
dir /S esptool.py

python <esptool-path>/esptool.py --port COM? erase_flash

Linux:

python <esptool-path>/esptool.py --port /dev/ttyUSB0 erase_flash

MacOS:

python <esptool-path>/esptool.py --port /dev/tty.usbserial-1420 erase_flash

Deploy the new firmware on ESP8266:

Download MicroPython image here:
Windows:

python <esptool-path>/esptool.py --port COM? --baud 460800 write_flash --verify -fs=detect -fm dio 0 \users\Marcus\Downloads\esp8266-20180511-v1.9.4.bin

Linux:

python <esptool-path>/esptool.py --port /dev/ttyUSB0 --baud 460800 write_flash --verify -fs=detect -fm dio 0 /home/marcus/esp8266-20180511-v1.9.4.bin

MacOS:

python <esptool-path>/esptool.py --port /dev/tty.usbserial-1420 --baud 460800 write_flash --verify -fs=detect -fm dio 0 /Users/marcus/Downloads/esp8266-20180511-v1.9.4.bin

Deploy the new firmware on ESP32:

Download MicroPython image here:
Windows:

python <esptool-path>/esptool.py --chip esp32 --port COM4 write_flash -z 0x1000 \users\Marcus\Downloads\esp32-20180730-v1.9.4-410-g11a38d5dc.bin

Linux:

python <esptool-path>/esptool.py --chip esp32 --port /dev/ttyUSB0 write_flash -z 0x1000 /home/marcus/esp32-20180730-v1.9.4-410-g11a38d5dc.bin

MacOS:

python <esptool-path>/esptool.py --chip esp32 --port /dev/tty.SLAB_USBtoUART write_flash -z 0x1000 /Users/marcus/Downloads/esp32-20180730-v1.9.4-410-g11a38d5dc.bin

Installing Libraries not included in MicroPython using upip and REPL:
- Connecting to Internet:
- import upip
- Use upip to install any library available on PyPI:

Testing:

List files on the board using Ampy, MFPShell or RShell:

ampy ls (needs AMPY_PORT env var)
mpfshell

open tty.SLAB_USBtoUART
ls

rshell

ls -l

Connect and interact with a MicroPython board using a serial connection.
Connect with the Device REPL using a terminal. The baudrate of the REPL is 115200 (Data: 8 bit; Parity: None; Stop bits: 1 bit; Flow Control: None):

Windows:

Use Putty or any other terminal software.

Linux:

ToDo

MacOS:

miniterm.py /dev/tty.usbserial-1420 115200

Press Enter and the terminal should show Python prompt like >>>

Try a Hello World:

print("Hello World")
import esp
esp.flash_size()
import uos
uos.listdir()

Blink Example:

from machine import Pin
import utime as time
led_builtin = 2 # D4=gpio02
led = Pin(led_builtin, Pin.OUT)
while True:

led.on()
time.sleep(1)
led.off()
time.sleep(1)

Additional Information:

You can use the Arduino IDE to upload your C/C++ software to the ESP boards, even after the upload of the MicroPython firmware. You don't need to upload the original Expressif default firmware to use the Arduino IDE. However, if you really wants to upload the Expressif firmware, then these as the instructions for reverting board to the original Expressif default firmware to use with arduino C/C++ language:

Download ESP Flash Tool
ESP8266

Download and extract the latest firmware (for ESP8266)
Execute ESP Flash tool and select the right board: (e.g., <downloads>/flash_download_tool_v3.6.4/ESPFlashDownloadTool_v3.6.4.exe)

ESP8266_NONOS_SDK-master/bin/blank.bin at 0x7E000 and 0x3E000
ESP8266_NONOS_SDK-master/bin/esp_init_data_default_v08.bin at 0x7C000
ESP8266_NONOS_SDK-master/bin/boot_v1.7.bin at 0x0
ESP8266_NONOS_SDK-master/bin/at/1024+1024/user1.2048.new.5.bin at 0x1000

ESP32:

ESP32 and ESP8266 - Configuring Arduino IDE and Updating firmware

Choose the right COM port
Choose Baud Rate 460800
Click "Start" button.

Tools:

EsPy IDE - for Windows - Micropython IDE for ESP8266
uPyLoader - for Windows - File transfer and communication tool for MicroPython boards
Adafruit MicroPython Tool - for MacOS, Linux and Windows - Utility to interact with a MicroPython board over a serial connection.
MPFShell - A simple shell based file explorer for ESP8266 and WiPy Micropython based devices - for MacOS and Linux
RShell - Remote MicroPython shell - for MacOS and Linux

References:

Friday, April 27, 2018

Installing MongoDB on Ubuntu 18.04 32bits

Ubuntu 18.04 Desktop
MongoDB Community Edition 3.2.19

Goals:

Install a mongodb server in Ubuntu 18.04 32bits.

Install:

curl -O https://fastdl.mongodb.org/linux/mongodb-linux-i686-3.2.19.tgz
tar -zxvf mongodb-linux-i686-3.2.19.tgz
mkdir -p ~/mongodb
cp -R -n mongodb-linux-i686-3.2.19/ mongodb
export PATH=~/mongodb/bin:$PATH
Starting MongoDB with previous config file and legacy storage engine. This way the databases from older version still active and with data already saved.
sudo ~/mongodb/bin/mongod --storageEngine=mmapv1 --config /etc/mongodb.conf

Testing:
Troubleshooting:

Error: solved Cannot start server. The default storage engine 'wiredTiger' is not available with this build of mongod.

force mmapv1 engine with option mongod --storageEngine=mmapv1

References: