Things I’ve learned so far this week (6 Dec)

It’s been a very long time since I’ve done one of these. So I thought I’d share. Not all of this was learned in the last week, but in the recent past.

  • When you do port forwarding with SSH, the default binding address for the forwarded port is 127.0.0.1. This is a security feature that prevents that port from being used by outsiders.
    • If you want to expose this, enable the configuration GatewayPorts in the SSHD configuration
  • Groovy has special DSL support for Cucumber. It makes writing cucumber steps easier and simpler to manage environment hooks
  • The obvious one, but rarely done in practice: Creating a POC in code is a lot better to get your core logic working than building out a full app and trying to make decisions later.
  • Bash: Substring extraction is possible in a very concise and easy manner. (Like most things in Bash, there are magical characters for that) It makes sub-arrays possible.
  • Bash function parameters are errrr different. Unlike most C-like languages, they define them by an array index than a variable name. (I.e. $1, $2, …)
    • You should define your local variables in your function near the top
    • All variables are global unless you declare them as local in bash

Other

  • Magic shows can be pretty cool (Thanks, Michael Carducci! [NFJS])
  • I need to see more firebreathing shows. (Rammstein included lots of fire displays)

Secure Services – It’s not just for REST or SOAP anymore

A note: I meant to write this blog post for years. I am finally knocking this blog post off of my todo list. Hurray!

In the beginning, there was the terminal, and it was great. After networking had come around, the terminal was networked and called telnet. It was thought to be good but later realized to be a disaster when people realized it was not secure. Then there came the secure shell which was a secure wrapper around telnet. Later the features for SSH expanded into:

  1. Port Forwarding (used as a jump box/service)
  2. Proxying network traffic
  3. X11 forwarding
  4. Secure shell environment
  5. Key based authentication
  6. Secure File Transfer (SFTP)

There are a lot of different uses of SSH and how you can configure it to do some pretty extraordinary things. That is something that would be out of the scope of this blog post. A great reference on SSH can be found in this book.

One of the features that caught my attention is that it is possible to create services that are purely in the Unix environment and are incredibly secure. The attack surface is small, communication is encrypted, and that your environment is sandboxed (well as much as you make it).

Authorized Keys

Passwords are incredibly low effort to unlock a system. They tend to be short, and they can be brute forced. (Even worse, they frequently have a small space of combinations as that they are human chosen). Randomly generated keys with lots of bits were created to avoid this issue. This was added to SSH to add in passwordless login and to avoid sharing passwords as well. All of the public keys are stored in the user’s authorized_keys file.

Within the authorized_keys, each entry has the following format:

<key encryption used> <public key> <comment>

Within each line, it is possible to extend the features (shell, a command to run, environment variables, etc.) of that particular login. (See the man page for more details)

To build a secure service, use the command variable. You will see in the section labeled your first secure service for an example.

The setup

  1. To setup an account, you are going to need a private key. Generate that with ssh-keygen without a password. (You can use a password, but it will make automation tough)
  2. Add the entry into the authorized_keys file. (~/.ssh/authorized_keys) With the ssh-copy-id command (ex. ssh-copy-id –i [location of your private key] [user running the command]@[server]
  3. Your ~/.ssh/authorized_keys file should have entries similar to the following:
    1. cat ~/.ssh/authorized_keys
    2. ssh-rsa A……..iJu+ElF7 steven@server
      1. See the section Authorized Keys section for an explanation of what this means.
  4. Test the access to the service by sshing into the box under that user and with that key. (Sample command: ssh –i [private key] user@server
    1. The first time you connect to a server with SSH you are going to get a message asking if it is ok to connect to a particular box. (This is something you need to do if you are automating a process as well)

Your first service

Your first service will be an incredibly simple example. Open up your authorized_keys file that was modified from the setup section. Add in the following command in front of the ssh-rsa/dsa line for the new entry.

command="echo hello the time is `date`"

The line of the login should now look similar to the following:

command="echo hello the time is `date`" ssh-rsa A.......

Now you can make a call to the configured service as such:

ssh -i .ssh/id_sitetest steven@localhost

You will receive an output of:

hello the time is Fri Nov 11 22:28:37 CST 2016
Connection to localhost closed.

Congratulations, if you followed along with the previous instructions: you have created your first secure service. All of the input communicated to the service and coming back from the service is encrypted. You have control over the format that is output and how you are going to take in input. The beauty of this service is that a terminal is not left open, and only the command that is defined is run. Once the command is done, the session automatically closes.

Suggestions

How should one best develop services?

You should develop a bash script to replicate the functionality that you would like the server to perform before setting it up to run in SSH. This gives you an isolated environment to test the process and to test it before it goes live. The SSH service setup is merely a layer above the script.

How should I get input into the service?
To do this, you would need to take in input just like how you would in a bash script. I would suggest using the ‘read’ command to do this. See this guide.

A note on this: Always validate the input. Attackers are unlikely (assuming that the key is managed properly), but it never hurts to

Is it webscale?

Honestly, I do not know the answer to this question. I guess it is possible to do this via the web, and I am not sure how stable it would be with lots of concurrent users. It is at most as scalable as the SSH service and system.

I did a brief search to see if this was possible in javascript on the client side and I could not find a source to show that it was possible.

Can you automate the use of these secure services?

Yes. However, when you create the key, you should never add a passphrase as that requires manual interaction. A word to the wise, keys should have a required live cycle and should be phased out periodically. Key management will be an issue in this situation.

Part 2 of 2: How To Make Your Life Easier As A Tech Worker (IT/Software Engineering/System Administrator): Motivation

The lack of motivation can affect your work and personal life severely. These are some of the things I found to be helpful:

  • Having a daily status. This helps you to understand what needs to be done, what needs to be done later, what problems you faced, and helps communicate what you’ve been up to with your peers/clients/superiors. Good daily statuses answer the questions:
    • Did you have any roadblocks today?
    • What did you do today?
    • What is on your task list today?
    • Also I find that its incredibly helpful to finish the daily status prior to leaving for the day and reviewing the items left to do in the morning when I get in. Once the daily status is reviewed it is incredibly easy to focus and knock out tasks.
  • Daily statuses may help you get a raise or help to establish job security. Daily statuses can help you to support your claims of higher pay due to work performed. With a daily status, your manager can go back through them and see work performed, and it’s easier to account for your work. My coworker uses this template. [As mentioned in the last part of this series].
  • Plan out the tasks that need to be performed prior to starting them. If you have a task to build a twitter clone, breaking the tasks into extremely small tasks such as evaluate database, create test interface, create authentication system, etc would be a good way to go about it. Reviewing prior to the start of work is the way to go about this. With a good understanding about the tasks to complete, it becomes incredibly difficult to procrastinate or avoid working. The tasks are easily achievable and you have something to show for it, even when the work isn’t incredibly visible.
  • Organize similar tasks in batches. I’m sure Tim Ferris wasn’t the first person to suggest this, but I found this tip from one of his books. If you have lots of individual tasks that have the same process, collect all of the tasks and perform all of the actions in a batch-like fashion. For example: If the task is to go to the store every time you need something, consider creating a list and going to the store to get all of the items at once. This will cut down on the costs (time, money, and stress) involved with driving to and from [his example was batching bill paying].
  • Monitor your time – this may be an extension of the daily status suggestion. Monitoring your time helps you to identify times where you may become unproductive or easily distracted. It also helps you identify the amount of time that you can work until you need a break.
  • Weekly self reviews – this involves a bit more time. A weekly self review can help you see the big picture of the tasks that you performed that week, and may even change your mind about the approach taken. For example, if you repeatedly struggled with a Java GUI component every day of the week, a weekly review would show that a different component might be beneficial or that it might be helpful to go for training on the individual component.
  • Clear out your email box. Similar to a cluttered desk, a full inbox makes emails difficult to search through, and may cause new emails to be lost.
  • Keep a notebook or journal. New ideas rarely come when you want them to. Keeping a physical notebook helps you to write the ideas out without becoming distracted with what you’re working on. Also, a physical notebook acts as a unique outside task from your current job. It’s similar to “Rubber Ducking” the problem. Roald Dahl suggested doing this, too – he kept a notebook and wrote down every idea he had for a story as soon as it came to him (even using the dust on his bumper once when his notebook was missing).
  • Keep A Blog: Blog about the problems you’ve faced and how you overcame them. For example, when I had an problem with a particular issue with my Gentoo configuration, I wrote a blog post after going through the trouble of finding a solution and testing many others. When I ran into the same issue again when re installing  the solution now was quickly available to me. Google picked up the post and made the search even quicker. Also, it may turn out to be an issue that others have faced. If your blog is fairly popular you may even get feedback, or kudos for solving the problem that others have had.
  • Writing down expectations/concerns about tasks. This may sound silly, but writing down the potential negative feelings you have towards a technical approach may help you overcome biases or aggravation down the road. http://life-sucks.org/negative-thoughts
  • Keeping a task list. Creating/using a task list will give a quick feedback on completing tasks. Use an online to-do manager that can schedule tasks, organize them into projects, and can integrate into your existing PIMs [Outlook, Kontact, Gmail+Calendar]. Some examples of this are doit.im, trello, vitalist, Toodledo, Nozbe, and Nirvana for GTD.
  • Keep track of things you want to learn. This helps when you have free time during work and want to learn something. Also, learning new technologies helps you with new tech projects and makes you more marketable.
  • Read Regularly. This may not be directly beneficial to completing the problems you face today, but this is more of a multiplier. It helps you to branch out from your current set of problems, and find new solutions. Blogs, RSS feeds, technical books, and most other books will help keep you motivated and productive.

Part 1 of 2: How To Make Your Life Easier As A Tech Worker (IT/Software Engineering/System Administrator): Automation

After some time of being a student, and a tech worker [Software Engineer, IT Consultant, or  System Administrator], you should start to find ways to make your job easier.  These are some of the things I found helpful:

  • Create templates for daily status emails. If your employer/ client requires a daily status, this makes the task of writing up the daily status easier. Outlook has the ability to keep track of email templates.
    • If the daily status is being sent to multiple people it may be helpful to create a distribution list. A distribution list is less likely to lose contacts between statuses.
    • One template a coworker of mine uses is this template.
  • Create filters for emails. This can help reduce the amount of email that requires attention. For example if a group communicates with a shared distribution list then create a rule or a filter that can organize that into a separate folder. This can also clear out “Build succeeded” emails [if your company automatically subscribes you to those]
    How To Create Rules In Outlook.
    How to Create GMail Filters.
  • Use SSH keys: SSH Keys can save a little bit of time by creating password-less logins. Also with SSH keys, scripts are able to run commands on other SSH enabled boxes. This means that much of the repetitive system administrative tasks can be automated. Gentoo Guide to SSH [This include generic non-gentoo instructions.
  • SSH Commands: SSH Commands are shell-less sessions in which a user logs into a SSH box with a key to kick off only one specific command. For example, say you had to reset an environment, and it required quite a few commands. With an SSH command, you would combine all of those commands and then kick off the command as an SSH command with its own key. From there on, you could kick off those commands by connecting to the SSH box with the key.  More information.
  • Backups- It can’t hurt to backup your data. I’ve never found a single person that was reliable at doing this frequently. Setting up rsync, SSH-keys and making it a cron job is a good first step. I’ve heard good things about luckyBackup, but I haven’t had a chance to try it. 
  • Using Salt/Puppet to automate system administrative tasks on multiple machines. This would be a bit more helpful for those who regularly roll out new software to multiple machines, or need to perform other system admin tasks.
  • IfThisThenThat (Review) is an online service that connects other online services to perform the intended result. For example, if a Google search reveals specific keywords, you can have that update a twitter account.

SSH Mastery, A Book Review

“SSH Mastery: OpenSSH, PuTTY, Tunnels, and Keys” by Michael Lucas is one of those technical books that you wouldn’t keep on your bookshelf. It’s one of the books that will have its bindings bent, and many pages bookmarked sitting near the keyboard. Well, that is until most of the information is second nature to you. This book is one of the rare books that lives up to its title, is short and concise, very valuable, and is rather inexpensive.

The author could approach the material as a dull professor, or regurgitate the man page. However, the author addresses an audience of IT professionals as a professional himself. The book introduces the reader to the SSH toolset, goes over the clients (for Mac, Linux, and Windows), key configuration, various SSH configuration bits, a little over X forwarding, the configuration options, and how to do tunneling (enough to emulate a VPN connection).

Besides the writing style, and targeting the audience, I think that the strong points of this book are based on the subtle details. For example, instead of marketing SSH as a mysterious and obscure protocol that is “secure,” it’s described as a wrapping protocol. Many online tutorials and top X SSH tip blog-posts market SSH as the former. Another being, the author takes a side on the option to use the SSH v1 vs. v2 option. He strongly discourages v1, due to its flaws. Lastly, he answered a question or two that I did not realize I had about SSH. The questions were: How can you prevent man-in-the-middle? How can you improve performance? The answers being, key exchanges and multiplexing.

The concerns that I have for this book are rather minor. I would have liked the author to provide some data demonstrating the performance of the protocol. Example of performance could come from a comparison of speeds of file transfers [and different ciphers used], or gathering some evidence on how much cost is involved in using a terminal through SSH. The other criticism that I have is that the book did not go into detail on X forwarding. The reason given: It varies quite a bit from platform to platform and is subject to change. It would have been nice to get a review on the different X clients and protocols available.

New Equipment and Switching Workstations

Recently I got a sweet new laptop. After being stuck with laptops with cheap builds and parts, this is a welcomed new addition. When converting my previous workstation over to the new one, a few things came about some bad, and mostly good. The awesome bit about the new machine is going from a core duo to a Core i7 processor. My primary operating system has been Gentoo, so installing a new operating system requires a manual install, configuring a new kernel, and building all of the needed secondary applications.

Let’s start off with the annoyances:

  • Since both machines are Gentoo sharing the USE flags from Make.conf, and adjusting the new one to the new hardware included an outdated keyword or two. It turns out that Xvmc is outdated and by using an Nvidia card, one should use VDPAU. This was fixed by eliminating all mentions of Xvmc keywords.
  • IceTea Java requires printing [Cups], which CUPS require Java. This circular dependency was quite annoying. At the moment, I cannot recall how this was fixed. However, I believe it was fixed by forcing Portage to pick Oracle’s version of Java rather than IceTea. [I’m not sure how to set the preference prior to installing; I had to force/nodeps this one].
  • CrossOver Professional [at least the 10.3 version] requires the use of Python 2. If you have a new install, then Python 3 will be in place. To fix this: you’ll need to fix the Python 2 references in the installation directory. I found this guide to be helpful, although, it was a bit difficult to find. If the guide has been deleted, this is a copy from the site:
    $ cd ~/cxgames/bin
    $ sed -i -e "s|#![ ]*/usr/bin/python$|#!/usr/bin/python2|" *
    $ sed -i -e "s|#![ ]*/usr/bin/env python$|#!/usr/bin/env python2|" *
    
    $ cd ~/cxgames/lib/python
    $ sed -i -e "s|#![ ]*/usr/bin/python$|#!/usr/bin/python2|" *.py
    $ sed -i -e "s|#![ ]*/usr/bin/env python$|#!/usr/bin/env python2|" *.py
  • CrossOver Pro, despite having a 64bit version, requires multilib support. [Which was included, but for those who wish to be 64bit purist may find a little too late]. The kernel must include IA32 Emulation support [from the Executable file formats / Emulations section]. The CrossOver Pro documentation claims you should have ia32-libs installed. In Gentoo this is the equivalent to the app-emulation/emul-linux-x86-* libraries.
  • Nvidia Optimia – There is no fix for my laptop. It’s frustrating. Although, the Intel video card does quite well. I’ve tried to install bumblebee, but the configuration isn’t currently supported.

Some nice changes in the Linux environment/Gentoo:

  • The added features to the Linux kernel. It looks like better SSD Support has been added, awesome!
  • NetworkManager/NetworkManagement. I’ve struggled with a lack of dynamic network support a while. This should be included in the KDE meta-ebuild. Although, from my understanding, there is quite a bit of approval/support for WCID. NetworkManager has great desktop support clients, if the user unplugs an Ethernet cable; it tries to switch to WIFI.
  • If you have CrossOver Pro, and use the Bottle functionality. You can create mutually exclusive containers for applications. You can also export these bottles as archives and import them later. This is amazing! To bring over your copy of MS Office, you don’t have to deal with the trouble of recreating the environment, or reinstalling the dependencies.
  • LSOF- This can help you find what process is listening to which port. That can be helpful when tracing down rogue/bad installs of server components. Looking for the process that is responsible for port 90? Use the command lsof –i :90.
  • Adding SSH keys for passwordless logins. Prior to using this command, I always setup this functionality manually. Then I heard about ssh-copy-id. This command takes your default key [or one specified by an argument] and copies it to the right place at the target. The downside to this command is that it depends on quite a few defaults. [I.e. Default port of 22, default users, port following, and default keys] This is an annoyance that has a bit of good in it. Having a non-default SSH setup will require you to alias and setup a personal SSH config.