Linuxizing your Windows PC #2

Technology Tips

This article may currently be obsolete as it has not been updated since 2018.

Last week I presented how to setup basic Linux features on Windows through the Bash On Windows aka Windows Subsystem for Linux (WSL). You should be now able to run variety of Linux apps both command line and with GUI. Also, presented how to install Docker and Vagrant which are popular tools to isolate your host environment from development playground.

Let’s install essential tools for variety of modern software development technologies and get prepared for some hardcore coding battles!

Reading feedback from comments on various websites really helped to select what tools should I try to install and run in WSL. I’m not actively using all of the languages and technologies tested down below, so feel free to comment if something isn’t right or perhaps you have some suggestions.

The Part 2 of Linux’izing Windows will be focused mostly on Web, DevOps and fancy Machine Learning stuff. I have skipped the Desktop or Mobile development as it is already perfectly doable on native Windows via dedicated tools, languages and technologies. Ok, maybe with exception or Linux desktop development, but that should just work with the X Server regardless of using Qt, GTK, or any other toolkit. OSX or iOS? Well for that you’ll need a Mac anyways.

The following is the table of contents from Part 1 and Part 2, you can jump straight to your weapon of choice.

Part I

Part II

Sharing environment between Windows and WSL

Probably should have started by desrcibing this last week, anyway, here it goes.
So, you install the WSL on your Windows PC and then what?

Where is WSL installed?

Your Linux filesystem will be created in your user’s AppData directory, preciesly in C:\Users\your.username\AppData\Local\lxss. It is a hidden folder, inside you’ll find root, home and other standard Unix directories, you should avoid manual changes there from outside of WSL.

How to share files between Windows and WSL?

Guess, there are plenty of ways, I’ll show you how I do it.

By default WSL has access to your Windows drive, it is mounted in /mnt/c, for example your Windows home directory is C:\Users\your.username and in WSL this directory is available at /mnt/c/Users/your.username.

I keep all my projects in the projects folder in my Windows home directory C:\Users\cepa\projects. When you start bash you’ll get into your WSL home directory which is located in WSL filesystem, however you can create a symlink to your projects folder.

cd ~ln -s /mnt/c/Users/your.username/projects projectscd projects

Accessing other disks and network locations?

Normally on Linux you would directly mount a filesystem from a disk using the mount command and a Windows network location using Samba. In WSL it is a bit different though.

WSL filesystem uses DrvFs which seems to be a Microsoft’s invention to share filesystem between Windows host and WSL layer, and apparently, you can use DrvFs to mount other Windows drives and network locations.

You can have a look here for more details:
https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/

Quick how to. Let’s say you have a drive D: in your Windows PC and to mount it run:
sudo mkdir /mnt/d mount -t drvfs D: /mnt/d -o metadata,uid=1000,gid=1000,umask=22,fmask=111
Now you can cd /mnt/d and access files. That works also with network locations mounted in Windows as shared drives.
For network locations you have two options, first mentioned above, just mount it as a drive, second is mounting directly in WSL:
sudo mkdir /mnt/share
sudo mount -t drvfs
The df -h works too:

Getting basic Linux utilities and services

Lets get ready for upcoming coding battles!

First, you can install bunch of standard Linux tools which are often required while installing other software packages or are just handy.

sudo apt-get install -y \
  apt-transport-https \
  lsb-release \
  ca-certificates \
  build-essential \
  curl wget unzip zip tmux git

It will get you the GCC compiler, Git and bunch of other utilities. There are lots of other tools available and for most you can follow Ubuntu how to’s to install them.

Accessing your Windows PC over SSH

This one is pretty cool

With WSL you not only can use SSH to access remote Linux servers but you can actually run OpenSSH server in your WSL to access your Windows PC from outside! Just like any regular Linux box.

So, OpenSSH server on WSL:

Reinstall OpenSSH in WSL
sudo dpkg-reconfigure openssh-server
Edit SSH configuration, sudo nano /etc/ssh/sshd_config, and change the following parameters:
Port 2222
# ...
UsePrivilegeSeparation no
# ...
PasswordAuthentication yes
Please be aware that it allows to use password authentication which is not the best idea, if you need SSH server please read about SSH hardening and password-less authentication.
Open Windows firwall for a custom port (2222)
In Windows 10 you might have an SSH Server Broker Services running which occupy port 22, thus the change to port 2222. You’ll need to open Windows 10 firewall to allow incoming connections to your SSH server.
– Type WF.msc in the Start menu
– In the Windows Firewall with Advanced Security click on Inbound Rules
– Add a new rule for TCP 2222 and allow the connection
Restart SSH serwer in WSL
sudo service ssh --full-restart
SSH into your Windows PC over from a remote host
ssh your.wsl.username@your.windows.pc.ip -p 2222

Done, should just work.

LAMP stack for PHP development

PHP is an amazingly popular technology for creating various web projects. Many developers share the love-hate relationship with it due to its low entry and loads of crap code around and projects that are total mess from code perspective. WordPress anyone?

Before WSL times, you had basically two options, either use Linux and develop in PHP on a production like environment or struggle with WAMP bundle which is an acronym for Windows Apache MySQL PHP.

With WSL you can have a regular LAMP – Linux Apache MySQL PHP – setup on your Windows PC that is very similar to production Linux based environment where your PHP application is most likely to be hosted.

Apache HTTP Server with PHP 7

This is going to be a pretty standard setup, you can of course install Nginx+PHP FPM or other variations, up to you.

Install Linux packages for Apache and PHP
sudo apt-get install \
apache2 \
libapache2-mod-php7.0 \
php7.0 \
php7.0-cli \
php7.0-common \
php7.0-curl \
php7.0-dev \
php7.0-gd \
php7.0-json \
php7.0-mysql \
php7.0-opcache \
php7.0-xml \
php7.0-bz2 \
php7.0-intl \
php7.0-mbstring \
php7.0-mcrypt \
php7.0-zip
Enable additional Apache modules like mod_rewrite
sudo a2enmod rewrite
Restart Apache service
sudo service apache2 restart
Create sample PHP file, run:
sudo nano /var/www/html/phpinfo.php
and write:
<?php phpinfo();
Ctrl+X
Open http://localhost/phpinfo.php, you should get the PHP Info page.

MySQL Server

On Windows PC you can either have a MySQL server installed as a regular Windows service or inside WSL, here goes the WSL way:

Install Linux MySQL packages:
sudo apt-get install mysql-server mysql-client
Start MySQL service:
sudo service mysql start
Open MySQL command line client, run
sudo mysql
it should display a MySQL prompt, can type:
show databases;
which should print something like:
You can use MySQL Workbench and manage your WSL’s MySQL using it, use localhost:3306 for the connection.

Composer

Composer is a de-facto standard tool for PHP dependency management. You can easily install it in WSL, for more details visit https://getcomposer.org/download/ here’s the short version:

Install Composer
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
php composer-setup.phpphp -r "unlink('composer-setup.php');"
Make it global
sudo mv composer.phar /usr/local/bin/composer
chmod +x /usr/local/bin/composer
Run it

Symfony Project

So, far so good however, a simple phpinfo.php isn’t really a PHP project.
Let’s try something heavier, a lot heavier…

Symfony is probably the most popular enterprise-grade PHP framework of today. Typical boilerplate project is made of around 2800 files and weights around 8MB, that’s quite a lot of files to process, which makes it… a bit slow.

Details are available here https://symfony.com/doc/current/best_practices/creating-the-project.html but for sake of this blog post let’s do it quick, you can create a fresh Symfony project using Composer mentioned above:

Create a new Symfony project
composer create-project symfony/skeleton sftest
It will take a while to finish…
Run the built-in PHP test server
cd sftest
php -S 127.0.0.1:8000 -t public
Open http://localhost:8000/ in a browser on Windows, you should get Symfony project page.

MEAN stack for Node.js development

JavaScript and Node.js are the backbone of modern frontend development. With WSL you can easily install Node.js ecosystem and use just like on Linux or MacOSX.

Node.js v9 and NPM

For details you can have a look here https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions

Install Node.js on WSL
curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash -
sudo apt-get install nodejs
Avoid permission errors (EPERM) with NPM by setting NPM to use a folder owned by your user to install Node.js components and binaries
mkdir ~/npm
sudo npm config set prefix ~/npm
sudo chown -R $USER:$(id -qn $USER) ~/.config
Add new NPM location to your WSL’s PATH variable
echo 'export PATH="~/npm/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
Update NPM packages
npm install -g npm
Install common Node.js components (optional)
Once Node.js and NPM are installed you can install bunch of common Node.js utilities
npm install -g typescript bower grunt gulp less sass yarn webpack

So, it looks good but again, let’s try something heavier.

Angular in WSL

NPM is there so you can try with a new boilerplate Angular project.

Install Angular CLI
npm install -g @angular/cli
Well… it will download a lot of dependencies so be patient…
Create a new Angular project
ng new ngtest
Build and run Angular in Node.js server
cd ngtest
ng serve --host 0.0.0.0 --disable-host-check
Open http://localhost:4200/ on Windows, you should get a blank Angular 4 project page.

React in WSL

There seem to be a religious war between Angular and React devotees so I shouldn’t forget about the other pillar of frontend stack, again, with NPM it’s easy.

Install React’s creator
npm install -g create-react-app
Wait… again… a lot of dependencies…
Create a new React project
create-react-app reacttest
Build and run React in Node.js server
cd reacttest
npm start
Open http://localhost:3000/, interestingly React in WSL has triggered my Windows default browser to automagically open that link for me.

MongoDB in WSL

Well, I admit, this one was a bit of pain…
Generally you can install and use MongoDB in WSL, there are some issues with the way the MongoDB service is being run in WSL/Linux.

The following is based on the officiall how to https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

Instal Mongo Community Edition 3.6
sudo apt-key adv –keyserver hkp://keyserver.ubuntu.com:80 –recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list sudo apt-get update sudo apt-get install mongodb-org
Unfortunately, Mongo won’t start as a service, seems to be an issue with Systemd/Dbus, but you can run is the other way. You can start mongod as mongodb user using su:
sudo su - mongodb -s /bin/bash -c "/usr/bin/mongod --config /etc/mongod.conf"
Connect to Mongo server using Mongo client:
Optionally you can use external clients like Mongo Booster and connect to Mongo server from a Windows app.

Python, Django and Machine Learning

Python is a fabulous language. I hardly can think of any project that can’t be built with it, anything from web applications, through cloud, image processing to machine learning. You can do it all with Python and the language itself is super easy to learn.

Python has its versions available for Windows, however the power of this language lies in the thousands of libraries created by community. Many of these libraries are either dependant of native libraries available on Linux or require direct device access for some features and that seem to be an issue in many cases while using a native Python for Windows.

It seems WSL has solved some of these issues. Although, you won’t get direct access to devices like GPU for say Machine Learning – at least I’m not aware you can – however, with WSL you can use libraries that are difficult to install natively on Windows, including these for mentioned Machine Learning.

Python in WSL and Virtualenv

First, let’s install Python in WSL and additionally virtualenv to manage project specific environments. This way you can have multiple version of Python in your projects, like Python 2.7 and Python 3.

sudo apt-get install python python-dev python-setuptools sudo easy_install pip sudo pip install virtualenv

Now run Python to confirm it works:

python --version

Good, now can build some Python projects.

Django in WSL

As a starter, can try with a fresh Django project.

Create a new virtualenv with Python 3 for a Django project
virtualenv -p python3 djangotest
Activate virtualenv cd djangotest
source ./bin/activate
Install Django
pip install Django
Create new Django project
django-admin startproject myapp
Start Django
cd myapp
python manage.py runserver
Open http://localhost:8000/, should get the default Django app page.

Python Machine Learning in WSL

This section is based on information found in these two articles:

https://github.com/dzorlu/GADS/wiki/Guide-to-installing-machine-learning-libraries-in-python
https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/

Create and activate a new virtualenv
virtualenv pyml cd pyml source ./bin/activate
Install NumPy
pip install numpy
Install SciPy
pip install scipy
Install Matplotlib
sudo apt-get install libfreetype6-dev pip install matplotlib
Install Patsy
pip install patsy
Install StatsModels
pip install statsmodels
Install ggplot
pip install ggplot
Install Sciki-learn
pip install scikit-learn
Install Theano
pip install theano
Install Tensorflow
pip install tensorflow
Install Keras
pip install keras

Seems to be all right.

Could have installed them in a single pip instal … command, but wanted to make sure each or these works and catch potential issues.

DevOps tools for Amazon AWS

With WSL you can easily run DevOps tools to manage your cloud infrastructure.

Azure tools should work out of the box in Windows, right Microsoft?
So, let’s get AWS tools up and running.

AWS CLI in WSL

If you work with Amazon AWS Cloud you’ll definitely need the AWS CLI which is the official command line client to manage your infrastructure in AWS cloud.

You can easily install it with Python Pip, see the Python section first and install Python. When done just run the following.

sudo pip install awscli aws configure

Follow the official documentation to configure it https://docs.aws.amazon.com/cli/latest/userguide/installing.html

You’ll need an AWS Account with Access Key and Secret Key which you can set up in IAM in the AWS Web Console.

Terraform and Packer in WSL

Ever heard of the Infrastructure as Code concept? Basically you use a meta language to describe how your infra should “look like” and that meta description is processed by some software that eventually will build up your servers, networking, storage, etc.

If you look for a new hobby in life you can use AWS CloudFormation and write ass long JSON objects to describe how you want your AWS infra to be.

But, it’s 2018 and there are better tools. Apparently, HashiCorp heard the moaning of DevOps hordes and inventend two tools Terraform and Packer to automate cloud operations. They aren’t perfect, but A LOT better than JSON and CloudFormation.

You have two options on Windows, you can use native Terraform and Packer builds for Windows or you can use Linux ones in WSL. It’s a blog post about WSL so I’ll use the Linux ones.

Install Terraform from https://www.terraform.io/downloads.html
Download the Linux 64bit version
Move it to the bin directory:
mv terraform /usr/local/bin chmod +x terraform /usr/local/bin terraform -v
Install Packer from https://www.packer.io/downloads.html
Download Linux 64bit version
Move it to the bin directory:
mv packer /usr/local/bin chmod +x packer /usr/local/bin packer -v

Sample output:

Things that suck

Hey Microsoft, you reading this? 🙂
Can you guys please improve a couple of things?

IO is slow, very slow!
I haven’t done any real benchmark yet but generally using WSL feels sluggish, a lot slower than working on Linux. Subjectively the difference is 10x or so which is a serious issue.
The problem is especially while working with technologies like PHP or Node.js where projects have many thousands of files and are processing them. On Linux that just flies, on WSL it’s often unbearably slow.
Frankly, I’m working on two computers now. One laptop with WSL on it which is my primary Office/Mail/Remote work machine and a normal stationary workstation with multiple monitors and Linux on it. Whenever I have a heavier task to do I feel more comfortable working from my workstation. To be fair, it would be good to perform a benchmark of WSL on a real workstation rather than a low power laptop but still I bet the IO speed difference would be significant.
Networking
Many sysadmin tools just don’t work. I guess that’s because the WSL kernel doesn’t have all the features and have to share networking with Windows. If you’re primary workload is development WSL is just all right, but if you mostly do network operations then you’d be way better on Linux or Linux VM.
Lack of Systemd/Dbus and system wide startup
See the MongoDB bit of this article. Linux packages usually contains the software and the startup scripts. Often, the startup scripts depend on Systemd which is not supported(?) in WSL and that causes issues with starting Linux services. That being said, be aware that you need to keep at least one Bash on Windows instance running to keep your services like say Apache, MySQL or SSH alive.
Filesystem permissions
Bash on Windows has its own filesystem embedded into Window, however it is not fully compatible with typical Linux filesystem. You might have issues while working with Node.js or some other toolkits that need setfacl to work properly for various users other than your own which owns the current bash process.
Mounting external filesystems
DrvFs works but can’t mount things like NFS or even SSHFS via Fuse. That would be nice to have.
Direct device access
When you go to /dev there are only some dummy devices, can’t access drives or other hardware. Not sure if that would be possible to implement though.

Hot or not?

Am I going to ditch Linux completely?

No, at least not yet.

WSL is a great improvement over working on either a raw Windows 10 or Windows with Cygwin however there are times that you need a real not a castrated Linux like WSL. I think the 80/20 rule applies to it as well. For a regular daily use you can perform most of your tasks within WSL, but whenever you need some more advanced networking, device access or performance having a real Linux workstation is just better.

Personally I think WSL shines by bringing long missing features to Windows, it allows you to get a flexible machine both for office and development workstation use. With release of the build 1709, Windows 10 has became a real alternative for Mac OS X as all-in-one environment for developers and perhaps in time if it gets event better it might be a serious alternative for Linux desktop workstations.

Time will tell.

Linuxizing your Windows PC #2

Sharing environment between Windows and WSL

Getting basic Linux utilities and services

LAMP stack for PHP development

MEAN stack for Node.js development

Python, Django and Machine Learning

DevOps tools for Amazon AWS

Things that suck

Hot or not?

Lukasz Cepowski

Related Articles

GPXLAB v1.0

Moo

GPXLAB