Lesson 6: Packages, Software, Libraries

Homepage Content Slides Video

Warning

This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill out our General Feedback Survey.

Software

Everything that isn’t hardware.

Sofware is what makes modern computers useful. When you install a piece of software you can then invoke that program using your computer to do a useful task.

Software is everything running on a compuer, from web-browsers to the operating system itself.

  • Code that is run on a Computer.
  • Binaries.
  • Scripts.

Libraries

Libraries are pieces of code that are used by multiple programs, but which are not usually run on their own.

For Example: Your OS might come with a “MathFunction” library for a given programming langauge since math is something almost every program ends up doing at least a little bit.

  • Often used to make development easier.
  • Rarely run on it’s own.
  • Shared code.

Package Management

Package Managers are the tool used to manage Software and Libraries (together called Packages) in your OS. Some are very easy to use, some work very well, and none of them are perfect.

  • Automagically manage software and libraries on your system.
  • Examples:
    • Android Play Store
    • Apple App store
    • Steam

Core Package Management Functionality

TLDR: To take care of installation, removal, and updates of software.

While all package managers don’t have every feaure listed below, most of them end up accomplishing most of these tasks out of necessity.

Install, upgrade, and uninstall packages easily.

Below is a hypothetical package manager carrying out these operations:

$ pkg-mgr install firefox
Finding [firefox] on pkg-mgr servers...............................
Downloading [firefox]..............................................
Are you sure you want to install [firefox]? (Y/n) y
Installing [firefox]...............................................
Cleaning up........................................................
$ pkg-mgr upgrade firefox
Checking for updates to [firefox]..................................
Packages [firefox] are up to date. Nothing to do!
$ pkg-mgr uninstall firefox
Are you sure you want to un-install [firefox]? (Y/n) y
Removing [firefox].................................................
[firefox] successfully uninstalled!

The package manager took the name of a program you wanted, found it on some trusted servers, downloaded the package, and placed the downloaded files in the correct place. It seems obvious but they weren’t always around, especially not in their current form.

Resolve package dependencies.

Resolving dependencies is when you download all of the packages another package needs to run.

  1. Program foo and program bar both need library baz.
  2. You install foo first.
  3. Your package manager installs baz.
  4. You then install bar.
  5. Your package manager checks that it already has the baz dependency.
  6. It installs bar without installing baz again.
Install packages from a central repository.

Most package managers download packages from a pre-defined set of trusted servers. This is for security reasons related to trust.

Plus – how else would it find the packages? Google?

Search for information on installed packages and files.

You can usually search all packages and descriptions for a given string. For instance:

$ pkg-mgr search python
Search Results:
- python2   : The python programming language (version 2)
- pyhton3   : The python programming langauge (version 3)
- virualenv : A program for managing python virtual environments.
Download and install pre-built binaries (usually).
Package managers used to download the source code for a program and would compile the code locally. Now packages are built once on a server and the binaries (which are much smaller) are downloaded to your computer.
Find package that provide a required library or file.
This is like reverse-searching an image and it can be useful when developing software. Very useful when you’re trying to find a dependency during development or while building a program from source.

Programming Langauge Package Managers

These are also package managers for programming languages! They can be used to download packages developed in a specific language, but mostly they are used to provide libraries and tools for a specific language.

Examples:

  • Python: pip
  • Ruby: gem, rubygems
  • Haskell: cabal
  • NodeJS: npm
  • ... and so on forever ...

Other Package Managers

Below are a few more notable/interesting package managers.

Portage
The Source-based package manager for Gentoo.
Yaourt
An Arch User Repository wrapper for Pacman, the Arch Linux Package manager.
Nix
A ‘Fully Functional/Transactional’ package manager.
Brew
An Open Soruce package manager for OSX.
Chocolatey
A package manager for Windows.

Installation from Soruce

Despite the fact that we have package managers sometimes they aren’t an option for package installation. The package may not registered with the package manager, we want to use a newer/older version of the package, or we want to develop with a custom version of the package. When this happens we have to install from source. This is scary until you do it once or twice and then you realize it’s not that bad.

How to install a package from source:

  1. Download source code .zip (Zip) or .tar.xz/bz2/xz (Tarball)

    First you need to obtain the source code. This is done by downloaded an archive of the code, which is basically a way to put the code in a box so it can be transferred as a single file.

  2. Unpack the downloaded code.

    Once you have the soruce code you need to unpack it. This is either using the tar program or unzip program depending on the type of archive.

  3. Run the setup and configuration scripts.

    These scripts can run a series of checks and configuration from detecting the OS you’re running to make optimizations all the way to yelling at you to manually install a dependency.

    If you’re not sure what setup and config scripts to run, check the source code’s README file. It should have one. If not, try running make config, make, and make install. If all else fails look up “How to install <package name>” or “How to install a <package langauge> from source”.

  4. Build the program.

    Just like in programing class, when you run gcc myfile.c -o a.out you will be building a binary from all of the soruce files you downloaded. Unlike class you will be running a wrapper like make or build which will run many commands for you automagically.

  5. Resolve any unmet dependencies and repeat last two steps until it works.

    This is by far the most painful part of the manuall installation process. If the install fails it will hopefully tell you why and hint at any libraries or external packages you need to install. Thankfully you can still use your package manager to install dependencies.

  6. Place the binaries in a consistent location.

    You’ll need to place binaries (have the +x bit flipped) in a place that your shell can find them. You should place it in a directory in your PATH environment variable.

These are the steps that a from-source package manager follows, you just have to do the by hand.

Using grep as an example:

$ wget http://mirrors.kernel.org/gnu/grep/grep-2.25.tar.xz
$ tar -Jxvf grep-2.25.tar.xz
$ cd grep-2.25
$ ./configure --prefix=$HOME/bin/
$ make
$ make install

TODO: Install sl

  • Install the git, gcc, make, ncurses-bin, ncurses-base, libncurses5-dev, and libncurses5-dev packages via package manager.
$ sudo apt install git gcc make ncurses-bin ncurses-base libncurses5-dev libncurses5-dev
[...]
  • Install sl from source into the directory ~/bin/.
$ git clone https://github.com/mtoyoda/sl.git
[...]
$ cd sl
$ make
gcc -O -o sl sl.c -lncurses
$ mkdir ~/bin
$ ln sl ~/bin/
$ echo "export PATH=$PATH:$HOME/bin" >> ~/.bashrc
$ source ~/.bashrc
$ whereis sl
sl: /home/username/bin/sl
$ sl