This article may currently be obsolete as it has not been updated since 2018.
Last week I presented how to setup basic Linux features on Windows through the Bash On Windows aka Windows Subsystem for Linux (WSL). You should be now able to run variety of Linux apps both command line and with GUI. Also, presented how to install Docker and Vagrant which are popular tools to isolate your host environment from development playground.
Let’s install essential tools for variety of modern software development technologies and get prepared for some hardcore coding battles!
Reading feedback from comments on various websites really helped to select what tools should I try to install and run in WSL. I’m not actively using all of the languages and technologies tested down below, so feel free to comment if something isn’t right or perhaps you have some suggestions.
The Part 2 of Linux’izing Windows will be focused mostly on Web, DevOps and fancy Machine Learning stuff. I have skipped the Desktop or Mobile development as it is already perfectly doable on native Windows via dedicated tools, languages and technologies. Ok, maybe with exception or Linux desktop development, but that should just work with the X Server regardless of using Qt, GTK, or any other toolkit. OSX or iOS? Well for that you’ll need a Mac anyways.
The following is the table of contents from Part 1 and Part 2, you can jump straight to your weapon of choice.
Part I
- Prerequisites
- Installing Bash on Windows (WSL)
- Getting a decent terminal
- Running Linux desktop apps with X server on Windows
- Docker with WSL
- Vagrant with WSL
Part II
- Share environment between Windows and WSL
- Getting basic Linux utilities and services
- LAMP stack for PHP development
- MEAN stack for Node.js development
- Python, Django and Machine Learning
- Java and JVM ecosystem
- DevOps tools for Amazon AWS
- Things that suck
- Hot or not?
Sharing environment between Windows and WSL
Probably should have started by desrcibing this last week, anyway, here it goes.
So, you install the WSL on your Windows PC and then what?
Where is WSL installed?
Your Linux filesystem will be created in your user’s AppData directory, preciesly in C:\Users\your.username\AppData\Local\lxss. It is a hidden folder, inside you’ll find root, home and other standard Unix directories, you should avoid manual changes there from outside of WSL.
How to share files between Windows and WSL?
Guess, there are plenty of ways, I’ll show you how I do it.
By default WSL has access to your Windows drive, it is mounted in /mnt/c, for example your Windows home directory is C:\Users\your.username and in WSL this directory is available at /mnt/c/Users/your.username.
I keep all my projects in the projects folder in my Windows home directory C:\Users\cepa\projects. When you start bash you’ll get into your WSL home directory which is located in WSL filesystem, however you can create a symlink to your projects folder.
cd ~
ln -s /mnt/c/Users/your.username/projects projects
cd projects
Accessing other disks and network locations?
Normally on Linux you would directly mount a filesystem from a disk using the mount command and a Windows network location using Samba. In WSL it is a bit different though.
WSL filesystem uses DrvFs which seems to be a Microsoft’s invention to share filesystem between Windows host and WSL layer, and apparently, you can use DrvFs to mount other Windows drives and network locations.
You can have a look here for more details:
https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/
- Quick how to. Let’s say you have a drive D: in your Windows PC and to mount it run:
sudo mkdir /mnt/d
mount -t drvfs D: /mnt/d -o metadata,uid=1000,gid=1000,umask=22,fmask=111
Now you can cd /mnt/d and access files. That works also with network locations mounted in Windows as shared drives. - For network locations you have two options, first mentioned above, just mount it as a drive, second is mounting directly in WSL:
sudo mkdir /mnt/share
sudo mount -t drvfs
- The df -h works too:
Getting basic Linux utilities and services
Lets get ready for upcoming coding battles!
First, you can install bunch of standard Linux tools which are often required while installing other software packages or are just handy.
sudo apt-get install -y \ apt-transport-https \ lsb-release \ ca-certificates \ build-essential \ curl wget unzip zip tmux git
It will get you the GCC compiler, Git and bunch of other utilities. There are lots of other tools available and for most you can follow Ubuntu how to’s to install them.
Accessing your Windows PC over SSH
This one is pretty cool
With WSL you not only can use SSH to access remote Linux servers but you can actually run OpenSSH server in your WSL to access your Windows PC from outside! Just like any regular Linux box.
So, OpenSSH server on WSL:
- Reinstall OpenSSH in WSL
sudo dpkg-reconfigure openssh-server
- Edit SSH configuration, sudo nano /etc/ssh/sshd_config, and change the following parameters:
Port 2222
# ...
UsePrivilegeSeparation no
# ...
PasswordAuthentication yes
Please be aware that it allows to use password authentication which is not the best idea, if you need SSH server please read about SSH hardening and password-less authentication. - Open Windows firwall for a custom port (2222)
In Windows 10 you might have an SSH Server Broker Services running which occupy port 22, thus the change to port 2222. You’ll need to open Windows 10 firewall to allow incoming connections to your SSH server.
– Type WF.msc in the Start menu
– In the Windows Firewall with Advanced Security click on Inbound Rules
– Add a new rule for TCP 2222 and allow the connection - Restart SSH serwer in WSL
sudo service ssh --full-restart
- SSH into your Windows PC over from a remote host
ssh your.wsl.username@your.windows.pc.ip -p 2222
Done, should just work.
LAMP stack for PHP development
PHP is an amazingly popular technology for creating various web projects. Many developers share the love-hate relationship with it due to its low entry and loads of crap code around and projects that are total mess from code perspective. WordPress anyone?
Before WSL times, you had basically two options, either use Linux and develop in PHP on a production like environment or struggle with WAMP bundle which is an acronym for Windows Apache MySQL PHP.
With WSL you can have a regular LAMP – Linux Apache MySQL PHP – setup on your Windows PC that is very similar to production Linux based environment where your PHP application is most likely to be hosted.
Apache HTTP Server with PHP 7
This is going to be a pretty standard setup, you can of course install Nginx+PHP FPM or other variations, up to you.
- Install Linux packages for Apache and PHP
sudo apt-get install \
apache2 \
libapache2-mod-php7.0 \
php7.0 \
php7.0-cli \
php7.0-common \
php7.0-curl \
php7.0-dev \
php7.0-gd \
php7.0-json \
php7.0-mysql \
php7.0-opcache \
php7.0-xml \
php7.0-bz2 \
php7.0-intl \
php7.0-mbstring \
php7.0-mcrypt \
php7.0-zip
- Enable additional Apache modules like mod_rewrite
sudo a2enmod rewrite
- Restart Apache service
sudo service apache2 restart
- Create sample PHP file, run:
sudo nano /var/www/html/phpinfo.php
and write:<?php phpinfo();
Ctrl+X - Open http://localhost/phpinfo.php, you should get the PHP Info page.
MySQL Server
On Windows PC you can either have a MySQL server installed as a regular Windows service or inside WSL, here goes the WSL way:
- Install Linux MySQL packages:
sudo apt-get install mysql-server mysql-client
- Start MySQL service:
sudo service mysql start
- Open MySQL command line client, run
sudo mysql
it should display a MySQL prompt, can type:show databases;
which should print something like: - You can use MySQL Workbench and manage your WSL’s MySQL using it, use localhost:3306 for the connection.
Composer
Composer is a de-facto standard tool for PHP dependency management. You can easily install it in WSL, for more details visit https://getcomposer.org/download/ here’s the short version:
- Install Composer
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
php composer-setup.phpphp -r "unlink('composer-setup.php');"
- Make it global
sudo mv composer.phar /usr/local/bin/composer
chmod +x /usr/local/bin/composer
- Run it
Symfony Project
So, far so good however, a simple phpinfo.php isn’t really a PHP project.
Let’s try something heavier, a lot heavier…
Symfony is probably the most popular enterprise-grade PHP framework of today. Typical boilerplate project is made of around 2800 files and weights around 8MB, that’s quite a lot of files to process, which makes it… a bit slow.
Details are available here https://symfony.com/doc/current/best_practices/creating-the-project.html but for sake of this blog post let’s do it quick, you can create a fresh Symfony project using Composer mentioned above:
- Create a new Symfony project
composer create-project symfony/skeleton sftest
It will take a while to finish… - Run the built-in PHP test server
cd sftest
php -S 127.0.0.1:8000 -t public
- Open http://localhost:8000/ in a browser on Windows, you should get Symfony project page.
MEAN stack for Node.js development
JavaScript and Node.js are the backbone of modern frontend development. With WSL you can easily install Node.js ecosystem and use just like on Linux or MacOSX.
Node.js v9 and NPM
For details you can have a look here https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions
- Install Node.js on WSL
curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash -
sudo apt-get install nodejs
- Avoid permission errors (EPERM) with NPM by setting NPM to use a folder owned by your user to install Node.js components and binaries
mkdir ~/npm
sudo npm config set prefix ~/npm
sudo chown -R $USER:$(id -qn $USER) ~/.config
- Add new NPM location to your WSL’s PATH variable
echo 'export PATH="~/npm/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
- Update NPM packages
npm install -g npm
- Install common Node.js components (optional)
Once Node.js and NPM are installed you can install bunch of common Node.js utilitiesnpm install -g typescript bower grunt gulp less sass yarn webpack
So, it looks good but again, let’s try something heavier.
Angular in WSL
NPM is there so you can try with a new boilerplate Angular project.
- Install Angular CLI
npm install -g @angular/cli
Well… it will download a lot of dependencies so be patient… - Create a new Angular project
ng new ngtest
- Build and run Angular in Node.js server
cd ngtest
ng serve --host 0.0.0.0 --disable-host-check
- Open http://localhost:4200/ on Windows, you should get a blank Angular 4 project page.
React in WSL
There seem to be a religious war between Angular and React devotees so I shouldn’t forget about the other pillar of frontend stack, again, with NPM it’s easy.
- Install React’s creator
npm install -g create-react-app
Wait… again… a lot of dependencies… - Create a new React project
create-react-app reacttest
- Build and run React in Node.js server
cd reacttest
npm start
- Open http://localhost:3000/, interestingly React in WSL has triggered my Windows default browser to automagically open that link for me.
MongoDB in WSL
Well, I admit, this one was a bit of pain…
Generally you can install and use MongoDB in WSL, there are some issues with the way the MongoDB service is being run in WSL/Linux.
The following is based on the officiall how to https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/
- Instal Mongo Community Edition 3.6
sudo apt-key adv –keyserver hkp://keyserver.ubuntu.com:80 –recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
sudo apt-get update
sudo apt-get install mongodb-org - Unfortunately, Mongo won’t start as a service, seems to be an issue with Systemd/Dbus, but you can run is the other way. You can start mongod as mongodb user using su:
sudo su - mongodb -s /bin/bash -c "/usr/bin/mongod --config /etc/mongod.conf"
- Connect to Mongo server using Mongo client:
- Optionally you can use external clients like Mongo Booster and connect to Mongo server from a Windows app.
Python, Django and Machine Learning
Python is a fabulous language. I hardly can think of any project that can’t be built with it, anything from web applications, through cloud, image processing to machine learning. You can do it all with Python and the language itself is super easy to learn.
Python has its versions available for Windows, however the power of this language lies in the thousands of libraries created by community. Many of these libraries are either dependant of native libraries available on Linux or require direct device access for some features and that seem to be an issue in many cases while using a native Python for Windows.
It seems WSL has solved some of these issues. Although, you won’t get direct access to devices like GPU for say Machine Learning – at least I’m not aware you can – however, with WSL you can use libraries that are difficult to install natively on Windows, including these for mentioned Machine Learning.
Python in WSL and Virtualenv
First, let’s install Python in WSL and additionally virtualenv to manage project specific environments. This way you can have multiple version of Python in your projects, like Python 2.7 and Python 3.
sudo apt-get install python python-dev python-setuptools
sudo easy_install pip
sudo pip install virtualenv
Now run Python to confirm it works:
python --version
Good, now can build some Python projects.
Django in WSL
As a starter, can try with a fresh Django project.
- Create a new virtualenv with Python 3 for a Django project
virtualenv -p python3 djangotest
- Activate virtualenv cd djangotest
source ./bin/activate
- Install Django
pip install Django
- Create new Django project
django-admin startproject myapp
- Start Django
cd myapp
python manage.py runserver
- Open http://localhost:8000/, should get the default Django app page.
Python Machine Learning in WSL
This section is based on information found in these two articles:
https://github.com/dzorlu/GADS/wiki/Guide-to-installing-machine-learning-libraries-in-python
https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/
- Create and activate a new virtualenv
virtualenv pyml
cd pyml
source ./bin/activate - Install NumPy
pip install numpy
- Install SciPy
pip install scipy
- Install Matplotlib
sudo apt-get install libfreetype6-dev
pip install matplotlib - Install Patsy
pip install patsy
- Install StatsModels
pip install statsmodels
- Install ggplot
pip install ggplot
- Install Sciki-learn
pip install scikit-learn
- Install Theano
pip install theano
- Install Tensorflow
pip install tensorflow
- Install Keras
pip install keras
Seems to be all right.
Could have installed them in a single pip instal … command, but wanted to make sure each or these works and catch potential issues.
DevOps tools for Amazon AWS
With WSL you can easily run DevOps tools to manage your cloud infrastructure.
Azure tools should work out of the box in Windows, right Microsoft?
So, let’s get AWS tools up and running.
AWS CLI in WSL
If you work with Amazon AWS Cloud you’ll definitely need the AWS CLI which is the official command line client to manage your infrastructure in AWS cloud.
You can easily install it with Python Pip, see the Python section first and install Python. When done just run the following.
sudo pip install awscli
aws configure
Follow the official documentation to configure it https://docs.aws.amazon.com/cli/latest/userguide/installing.html
You’ll need an AWS Account with Access Key and Secret Key which you can set up in IAM in the AWS Web Console.
Terraform and Packer in WSL
Ever heard of the Infrastructure as Code concept? Basically you use a meta language to describe how your infra should “look like” and that meta description is processed by some software that eventually will build up your servers, networking, storage, etc.
If you look for a new hobby in life you can use AWS CloudFormation and write ass long JSON objects to describe how you want your AWS infra to be.
But, it’s 2018 and there are better tools. Apparently, HashiCorp heard the moaning of DevOps hordes and inventend two tools Terraform and Packer to automate cloud operations. They aren’t perfect, but A LOT better than JSON and CloudFormation.
You have two options on Windows, you can use native Terraform and Packer builds for Windows or you can use Linux ones in WSL. It’s a blog post about WSL so I’ll use the Linux ones.
- Install Terraform from https://www.terraform.io/downloads.html
Download the Linux 64bit version
Move it to the bin directory:mv terraform /usr/local/bin
chmod +x terraform /usr/local/bin
terraform -v - Install Packer from https://www.packer.io/downloads.html
Download Linux 64bit version
Move it to the bin directory:mv packer /usr/local/bin
chmod +x packer /usr/local/bin
packer -v
Things that suck
Hey Microsoft, you reading this? 🙂
Can you guys please improve a couple of things?
- IO is slow, very slow!
I haven’t done any real benchmark yet but generally using WSL feels sluggish, a lot slower than working on Linux. Subjectively the difference is 10x or so which is a serious issue.
The problem is especially while working with technologies like PHP or Node.js where projects have many thousands of files and are processing them. On Linux that just flies, on WSL it’s often unbearably slow.
Frankly, I’m working on two computers now. One laptop with WSL on it which is my primary Office/Mail/Remote work machine and a normal stationary workstation with multiple monitors and Linux on it. Whenever I have a heavier task to do I feel more comfortable working from my workstation. To be fair, it would be good to perform a benchmark of WSL on a real workstation rather than a low power laptop but still I bet the IO speed difference would be significant. - Networking
Many sysadmin tools just don’t work. I guess that’s because the WSL kernel doesn’t have all the features and have to share networking with Windows. If you’re primary workload is development WSL is just all right, but if you mostly do network operations then you’d be way better on Linux or Linux VM. - Lack of Systemd/Dbus and system wide startup
See the MongoDB bit of this article. Linux packages usually contains the software and the startup scripts. Often, the startup scripts depend on Systemd which is not supported(?) in WSL and that causes issues with starting Linux services. That being said, be aware that you need to keep at least one Bash on Windows instance running to keep your services like say Apache, MySQL or SSH alive. - Filesystem permissions
Bash on Windows has its own filesystem embedded into Window, however it is not fully compatible with typical Linux filesystem. You might have issues while working with Node.js or some other toolkits that need setfacl to work properly for various users other than your own which owns the current bash process. - Mounting external filesystems
DrvFs works but can’t mount things like NFS or even SSHFS via Fuse. That would be nice to have. - Direct device access
When you go to /dev there are only some dummy devices, can’t access drives or other hardware. Not sure if that would be possible to implement though.
Hot or not?
Am I going to ditch Linux completely?
No, at least not yet.
WSL is a great improvement over working on either a raw Windows 10 or Windows with Cygwin however there are times that you need a real not a castrated Linux like WSL. I think the 80/20 rule applies to it as well. For a regular daily use you can perform most of your tasks within WSL, but whenever you need some more advanced networking, device access or performance having a real Linux workstation is just better.
Personally I think WSL shines by bringing long missing features to Windows, it allows you to get a flexible machine both for office and development workstation use. With release of the build 1709, Windows 10 has became a real alternative for Mac OS X as all-in-one environment for developers and perhaps in time if it gets event better it might be a serious alternative for Linux desktop workstations.
Time will tell.
Other links:
- Discussion on Hacker News (Part 1) https://news.ycombinator.com/item?id=16351716
- Discussion on Hacker News (Part 2) https://news.ycombinator.com/item?id=16453751
- WSL Benchmark by Phoronix https://www.phoronix.com/scan.php?page=article&item=wsl-february-2018&num=1
Feel free to post feedback in comments, social media, email, etc and thanks in advance for sharing this article.