Applications



Any time you hear of an application you'd like to try, first go to Start menu and see if it's already installed in your system. If not, go to Software Manager / Store and see if it's available there. [But check to see if the version you get is seriously old.] If not, do "apt list | grep NAME" and see if it's in the repo. Also "snap search NAME" and "flatpak search NAME". If not, go to web site for the application and get it from there.

Make sure you have all repos enabled.

Applications that work well and I use have a green check-mark next to them. Your needs and opinions may differ from mine.



Browser

+/-
Probably best to stay with major browsers; some minor browser may not get security updates very often.

Important: After you install a browser, disconnect from the internet, launch the browser for the first time, turn off telemetry and other features you don't want, quit, connect to internet again, launch the browser again.

  • green check-mark  Firefox:
    +/-
    Arnab Satapathi's "Speed up firefox with RAM cache and tmpfs in Linux"
    Firefox Support
    Firefox "about" special pages
    Keyboard shortcuts
    Dedoimedo's "How to make Firefox scrollbars permanent, thicker"
    blarg's useful features in Firefox
    Mozilla's "Profiles - Where Firefox stores your bookmarks, passwords and other user data"
    Bugzilla
    random neuron misfires' "Deploying Firefox and Thunderbird Policies"
    bolucat / Firefox (source code)

    Brady Gavin's "How to See (and Disable) the Telemetry Data Firefox Collects About You"
    about:telemetry
    about:preferences#privacy and scroll down to "Firefox Data Collection and Use"

    Nathan Sweaney's "Silencing Firefox's Chattiness for Web App Testing"
    Mozilla's "How to stop Firefox from making automatic connections"

    Dan's "Here's Why Firefox is Seeing a Continuous Decline for Last 12 Years"
    Madaidan's Insecurities' "Firefox and Chromium"

    Modifying the "chrome" of the browser:
    Dedoimedo's "How to customize Firefox UI - step-by-step tutorial"
    rafaelmardojai / firefox-gnome-theme
    FirefoxCSS Store
    Firefox's "Browser Toolbox"
    /r/FirefoxCSS

    Modifying the databases under the browser:
    Mumbling about computers' article

    Idea: I'd like to see a feature for displaying and controlling the extension hierarchy/stack:
    +/-
    I'd like to see in what order my extensions and the cache and page JavaScript get to see requests and responses as they go from user to network and back.

    For example, a request might go in this order:
    1. user
    2. Facebook Container extension
    3. Privacy Badger extension
    4. uBlock Origin extension
    5. cache
    6. network stack (including VPN)
    This would be useful to understand the "priority" of each extension, which gets first shot at each request/response, where a request was blocked or satisfied. That also might be important if one of the extensions was an encryption extension: which other extensions are seeing my traffic in plaintext, and thus have to be trusted ?

    Once we had the hierarchy, maybe then we could start to analyze and control it. "Tell me which layer blocked or satisfied this request." "Don't use this extension for this domain."

    It would be great if the hierarchy shown extended down into the network stack, to show if a VPN was active, for example.

    We might need a parallel hierarchy for DNS requests. A DNS request could be processed/modified in the browser (DOH), in a proxy or VPN browser extension, in a VPN layer in the network stack, in the OS DNS mechanism(s).

    9/2020: I changed from deb package of Firefox to snap version, on Ubuntu GNOME 20.04:
    +/-
    There also is a flatpak available. All three packages were almost identical version numbers.

    Backed up old $HOME/.mozilla tree.
    "sudo apt remove firefox".
    "snap install firefox".

    Firefox no longer appears in the icons on the left side of the desktop; have to search for it in search bar to launch it.

    To copy profile across, in new browser I went to "about:profile", created a new profile NAME, set it to be default profile, quit browser, then copied contents from inside OLDPROFILE dir into new random.NAME directory, replacing/merging as it copied. Deb package keeps profiles under $HOME/.mozilla/firefox, snap package keeps profiles under $HOME/snap/firefox/common/.mozilla/firefox.

    In Preferences, you will see "Your browser is being managed by your organization", which I assume means "updates will be done by something outside the application (snap updater)". https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/docker/firefox-snap/policies.json

    Bug: In Downloads list, clicking on folder icon to "open containing folder" does nothing. Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1664824

    Bug: In "Save Link As" dialog, download directory is strange, not remembered, and deleting a file does strange things. Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1665090

    Bug: "Save Page As" always results in download-failed.

    Bug: strange characters appear in buttons or text fields sometimes.

    Apparently FF snap uses a temp folder for downloads: $HOME/Downloads/firefox.tmp

    9/2020: I changed from snap version of Firefox to Flatpak version, on Ubuntu MATE 20.04:
    +/-
    There were two choices of image in the middle of the install: a "BaseApp" and another. I chose the other. What's the difference ?

    In Flatseal, added some local directories I want FF to be able to use. Add one directory per line; the help-example is a bit misleading. And you can add ":ro" after a directory name to make it read-only.

    When FF starts, it's not the default browser, and nothing I do will make that Preferences / "Make default browser" button work. On CLI, did "xdg-settings set default-web-browser org.mozilla.firefox.desktop" and that worked to make FF the default browser. But in Preferences FF still thinks it's not the default browser.

    Bad design: I have permissions set to deny access to $HOME/Documents, yet I can save files into there. See Disappointed with Flatpak security model section of my VMs page.

    Bug: in download manager, click on "Open in enclosing folder" which should be $HOME/Documents, file manager opens in pathname "/run/...".

    And now clicking on a link in VS Code (which is a snap) results in launching a new instance of FF (which is a Flatpak), instead of opening a new tab in the existing instance of FF.

    To write less to disk/SSD: in about:config, set browser.cache.disk.enable = false, browser.cache.memory.capacity = 409600 (KB), browser.cache.memory.enable = true.

    Firefox blocks about 80 ports that users shouldn't normally access. To stop blocking, in about:config, create "network.security.ports.banned.override" string, which can be of form "6000,7000-8000".

    To see a list of config values you've changed, go to "about:support" and scroll down to "Important Modified Preferences" section.

    To see a list of all categories of "about" commands, go to "about:about".

    mozillaZine's "about:config entries" (old)
    Config source code: all.js
    Config source code: firefox.js
    Config source code: StaticPrefList.yaml

    To see all cookies, something like:
    
    find ~/.mozilla/firefox -name "cookies.sqlite" -exec sqlite3 {} "SELECT * FROM moz_cookies;" \;
    


  • LibreWolf:

    "A custom version of Firefox, focused on privacy, security and freedom."
    Mainly, defaults to more private/secure than FF, but you can reverse those settings.
    LibreWolf
    Removes some features that I probably want: search and form history, crash report.
    Available as Flatpak.


  • Mullvad:

    A custom version of Tor/Firefox, which doesn't use Tor/onion network.


  • Chrome:

    Raj's "How To Install Google Chrome on Linux Mint 19 'Tara'"
    Profiles stored in $HOME/.config/google-chrome directory.
    Cache stored in ??? For Flatpaks, under $HOME/.var/app/*/cache
    Use "--disk-cache-dir" option to store cache on ramdisk ?
    Type Shift+Esc to open Chrome's task manager.
    Go to "chrome://about/" to see all special URLS.
    ThioJoe's "The Secret Google Chrome Menus" (video)

    To get dark mode, you can add another shortcut for the browser with "--enable-features=WebUIDarkMode --force-dark-mode" added to args.

    If Chrome says "managed by your organization", fix via something like "sudo dnf remove fedora-chromium-config".

    To see a list of all categories of "chrome" commands, go to "chrome://about/".

  • Chromium:

    Profiles stored in $HOME/.config/chromium directory.

    To get dark mode, you can add another shortcut for the browser with "--enable-features=WebUIDarkMode --force-dark-mode" added to args.

  • Thorium:

    Based on chromium engine. Can use Chrome extensions and Google Sync.
    Alex313031 / Thorium


  • ungoogled-chromium:
    +/-
    ungoogled-chromium
    Repology's "Versions for ungoogled-chromium"
    Have to uninstall Chromium first, if it's installed.
    Segmentation Fault's "A review of ungoogled-chromium patches"

    Note: Things such as extensions and certificate-revocation-lists won't get updated automatically by the app; I assume default extensions and revocation-lists will get updated when you update the whole app from Linux package manager. If you're not adding extensions, and are updating frequently via Linux package manager, maybe this is okay.

    After installing from deb, it appears in app menus as "Chromium Web Browser" and on CLI as "chromium" (same names as normal Chromium browser), which I find confusing. Have to do "apt show ungoogled-chromium" to verify that right thing is installed. Also "chromium --help" mentions "ungoogled-chromium" about 8 lines down from the top.

    If you install from Flatpak, appears as "ungoogled-chromium" ?
    "flatpak run com.github.Eloston.UngoogledChromium --version"

    Profiles stored in $HOME/.config/chromium directory.

    Certificates are not stored in profile; must be stored in OS's cert store. So when I distro-hop, I have to re-install all the certs.

    Uninstalling and installing the two browsers (Chromium and ungoogled-chromium), I ended up in some "your profile is newer than this browser supports" situation. Tried to re-install and ended up in hang situations in Software Manager. Tried "sudo apt install ungoogled-chromium" and got "Package 'ungoogled-chromium' has no installation candidate".

    Went to Ubuntu Downloads page clicked on "18.04 (bionic) amd64", clicked top/newest version, clicked last deb link (one that didn't have some particular tag such as common, shell, driver, etc), downloaded it. Tried to install it, found you have to download and install the "common" package first.

    Has moved to common and main

    After installed, the only way to distinguish it from full Chromium is to do "apt list | grep ungoog".

    Upgrading to new version: download DEB files, then
    
    sudo apt remove ungoogled-chromium
    sudo apt remove ungoogled-chromium-common
    sudo dpkg -i ungoogled-chromium-common*.deb
    sudo dpkg -i ungoogled-chromium_*.deb
    rm ungoo*.deb
    

    Can't install extensions from Chrome Web Store directly:
    1. In ungoogled-chromium, go to three-dots / More Tools / Extensions and turn on Developer Mode.

    2. Go to Victor-Bonnelle / extension-downloader follow instructions to download extension-downloader.crx file.

    3. Drag-and-drop that file onto the Extensions page in ungoogled-chromium. The extension should be installed.

    4. Go to to Chrome Web Store and find extension you want. Or click on some other site's link to the extension you want, to get to the right page in the store.

    5. Click on blue download-icon in browser's address bar (icon for extension-downloader extension). The extension you want should be installed.

    See FAQ.

    Also: NeverDecaf / chromium-web-store

    To see status of GPU settings, go to "chrome://gpu".

    To get dark mode, you can add another shortcut for the browser with "--enable-features=WebUIDarkMode --force-dark-mode" added to args.

    From someone on reddit 12/2022:
    Let's compare it to Brave. Ungoogled Chromium has:
    • Slower chromium security updates.

    • Missing the important component updater for out-of-band security updates! This also includes CRLsets and extension updates.

    • Missing mDNS support, which is important for WebRTC privacy.

    • Worse state partitioning.

    • Worse fingerprinting mitigations.

    • No sophisticated built-in content-blocker.

    • Lack of toolchain hardening (e.g missing CFI in linked libraries).

    • No official builds for Windows.





  • Edge:

    Based on chromium engine. Can use Chrome extensions.
    I haven't heard that it has any particular features I want.
    One interesting feature: a link between Edge's Developer Mode and VS Code, so you can hop between the rendered page element and the editor holding the code.
    MS article
    Matt Callahan article
    Martin Brinkmann's "Microsoft Edge submits nearly any visited page to Bing"


  • Iridium:

    Based on chromium engine; open-source; privacy enhancements.
    Iridium Browser
    bug tracker
    Seems to be a bit out of date; maybe few users.


  • Vivaldi:

    Based on chromium engine; open-source; privacy enhancements.
    "Vivaldi can use many browser extensions developed for Google Chrome and Firefox (they use common WebExtensions API), and users can install extensions directly from the Chrome Web Store."


  • Brave:
    +/- Based on chromium engine; open-source; privacy enhancements.
    Emphasis on blocking trackers.
    Optional "Brave Rewards" to earn frequent flier-like tokens for viewing privacy-respecting ads.
    Maybe use the beta version to get security updates more promptly.
    Brave Browser
    Brave Browser bug tracker
    On Ubuntu 20.04, not available in repos or snap store or flathub.
    On Kubuntu 20.10, available in snap store or flathub, but snap has directory-permission issues.

    Installing native: Installing Brave on Linux
    "brave-browser-beta --version"
    "man brave-browser-beta"

    In Settings/Appearance, I enabled "Hide Brave Rewards button", "Always show full URLs".
    In Settings/NewTabPage, I disabled "Sponsored images", "Brave Stats", "Brave Rewards", "Binance", "Gemini".
    In Settings/SocialMediaBlocking, I disabled everything.
    In Settings/Extensions, I disabled "Hangouts", "WebTorrent", "Widevine".
    In Settings/Advanced/PrivacyAndSecurity, I disabled "Send analytics to Brave", "Google Safe Browsing", "Allow sites to check for payment methods", went into "Clear browsing data" and set most to clear on exit, went into "Site and Shields Settings" and "Payment Handlers" and set to "do not allow".
    In Settings/Advanced/Autofill, I disabled everything.
    In Settings/HelpTips, I disabled "Show Wayback Machine prompt on 404 pages".
    In Settings/System, I disabled "Continue running background apps when Brave is closed".


    Check the settings again after updates; sometimes new sponsored sites are added, and enabled by default.

    Like most chromium browsers (I think), it shows downloads in a big bar at the bottom of the page which can't be suppressed. Very annoying. Install the extension "Download Shelf Autohide" and un-check "open enclosing folder when downloads finish".

    To see status of GPU settings, go to "brave://gpu".


  • Looking for a very minimal GUI browser:
    +/-
    • GNOME Web (Epiphany):

      This one does have printing, history, bookmarking, homepage setting, downloader, search engines, incognito mode, ad-blocking, popup-blocking, dangerous-site-blocking, password storage, Firefox sync, spell-checker. Most of this can be turned off.

      Able to play mp4's, do JavaScript, cookies, HTTPS, tabs. Has view-source and devtools.

      No add-on mechanism (so can't do tab containers, VideoDownloadHelper, form history control, To Google Translate).
      No support for digital certificates.
      No support for network configuration / proxy.
      No "clear all cookies when quit".
      New tab always starts with "most visited pages" icons, which I don't like. No way to clear that list, I think.
      Click on a link and new tab is created, but focus doesn't move to it.

      "sudo apt install epiphany-browser"
      "snap install epiphany"
      "flatpak install org.gnome.Epiphany"
      Project site
      Bug-reporting
      Mailing list
      Aral Balkan's "Enabling Better Blocker in GNOME Web"

      Used deb version for a while, then it crashed, and every time I launch it again, it opens the same state and instantly crashes again. Fixed with "rm $HOME/.local/share/epiphany/session_state*"

      But it continues to have fairly frequent problems: crashes, and unpredictable long delays in opening save-file dialog for one site (confirmed that it fetches image again when you do save-as).

      It does have devtools: ctrl+shift+I. Not shown in the menus.

      It has some undocumented key-shortcuts. Rightclick-T on a link to open it in a new tab. Rightclick-S on an image to do "Save image as".

      9/2020: installed Flatpak version.

    • Falkon (formerly QupZilla):

      Has incognito mode, bookmarks, ad-blocker, homepage setting, history, printing, download manager, devtools, search engines, "clear cache/storage/cookies when quit", extensions. Also "Sessions", profiles, password manager.

      No pop-up blocker. No support for digital certificates.

      Uses KDE-style Save dialog, not Ubuntu GNOME native style.

      If you expand size of SaveAs dialog, new size is not remembered for next time you do SaveAs.

      Uses a chromium engine in Qt, which often is old, perhaps hurting security.

    • Min:

      Have to download from https://minbrowser.org/

      Installed deb 1.14.1 on Ubuntu GNOME 20.04.

      Has bookmarks, ad-blocker, history, printing, devtools, search engines, password manager.

      No incognito mode, pop-up blocker, download manager, homepage setting, "clear cache/storage/cookies when quit", extensions. No support for digital certificates.

      Always (?) launches showing last-displayed page, no way to turn that off.
      Shows downloaded files in bottom status bar, no way to turn that off.
      When create new tab, doesn't switch to it immediately.

    • Cliqz:

      Installed snap 1.8.1 on Ubuntu GNOME 20.04.

      Has bookmarks, ad-blocker, tracking-blocker, pop-up blocker, anti-phishing, history, printing, extensions, incognito (forget) mode but it's selective (for adult sites only ?) not universal, devtools, homepage setting, search engines, digital certificates, (optional) integration with LastPass password manager.

      Unable to save images to a VeraCrypt volume, probably because of snap permissions. Tried "snap connect cliqz:removable-media :removable-media" and "snap connect cliqz:system-files :system-files" but neither worked.

      I'm told "Cliqz doesn't seem to be developed any longer. ... The Snap is a beta from 2017."

    • Dillo:

      Installed deb 3.0.5-6build1 on Ubuntu GNOME 20.04.
      Last updated in 2015.

      No Preferences dialog; use $HOME/.dillo/dillorc file.

      No support for frames or JavaScript, which will break some pages.

      Save dialog doesn't scroll through sub-dirs as you type chars, and as soon as you change directory it forgets original filename.

      Has bookmarks, history, homepage setting, ad-blocker (use $HOME/.dillo/domainrc file), search engines (can disable), cookies (use $HOME/.dillo/cookiesrc file; disabled by default).

      No printing, devtools, digital certificates.

      Runs two server processes, dillo/dpi/file/file.dpi and dillo/dpi/bookmarks/bookmarks.dpi, and they stay running after you quit the browser.

    • Midori:

      Installed snap v8.0-31-gf6b3b1e on Ubuntu GNOME 20.04.

      Setting home page to a local file doesn't work ? Opens "Speed Dial" (bookmarks) tab instead. Every new tab starts with that title too.

      Save File always saves only to default Downloads directory.

      This one does have bookmarking, history, homepage setting, (very bad) ad-blocking, spell-checker, extensions. Most of this can be turned off.

      No incognito mode, popup-blocking, support for digital certificates.
      No "clear all cookies when quit".


    • Netsurf:

      Installed deb 3.6-3.2 on Ubuntu GNOME 20.04.

      Has bookmarks, pop-up blocker, ad-blocker, homepage setting, history.

      No incognito mode, support for digital certificates.
      No "clear all cookies when quit".

      Used View menu to hide menu bar, then couldn't find a way to get it back ! No way to get to any of the menus. But quitting and launching again made it re-appear.

      Did a simple Google search, clicked on a link, and the browser crashed. Can do it again and again, crashes every time.

      Even though preference "switch to new tab immediately" is set, it doesn't do it.

      No right-click-save-image; you can view an image alone in a tab and then ctrl-S to save it.

      Saving an image didn't work ! It always either creates a directory of that name, or fails silently.

      Tried to subscribe to their Users mailing list, to report bugs, and got a "their IP is blacklisted" failure from some German email daemon.

    • Grab source of ungoogled-chromium, remove features, and build it:

      ungoogled-chromium


    • Servo:

      Not released yet.
      Servo, the Parallel Browser Engine Project

      Apparently can't be run unless
      
      glxinfo | grep 'compat profile'
      
      says at least 3.1; mine says 2.1.

    kweb: last updated in 2014. Not in Ubuntu repos.

    uzbl: last updated in 2016. Not in Ubuntu repos. https://www.uzbl.org/

    arora: last updated in 2017. Have to build from source.

    luakit: just a blank window, I guess it requires some UI app.

    Conkeror: last updated in 2019 ? Not in Ubuntu repos.

    links2: "sudo apt install links2". Saves files only to file dir specified by "download_dir" in $HOME/.links2/links.cfg (default: home directory) unless you type more path manually. Can't copy/paste URLs, but you could edit $HOME/.links2/bookmarks.html file. http://links.twibright.com/ No longer being developed.

    surf: Last updated in early 2019. "sudo apt install surf". Can't copy/paste URLs into GUI, but you can launch from CLI via "surf URL". Save fails with "execvp x-terminal-emulator failed: Permission denied". Various other errors in terminal window. http://surf.suckless.org/


One quirk: when saving files to a VeraCrypt volume, Firefox will forget the proper setting of "last directory saved to". GNOME Web (Epiphany) doesn't have that problem.

It seems that when you're browsing/reading text pages, you want "open link in new tab" to automatically switch focus to the new tab. But if you're going to download a bunch of photos from a photo-gallery page, you don't want that, you want focus to stay on the original page.

Bookmarklets / scriptlets:
Kowalski7cc's "Gain back the control of your browser"
To fill in fields on a page:

javascript:{
document.querySelector("[name=firstname]")[0].value="Firstname";
document.querySelector("[name=lastname]")[0].value="Lastname";
void(0)}


See "Test your browser" section of "Computer Security and Privacy" page

Browsers write a tremendous amount of data to disk.
+/-
To see the write volume, do "sudo iotop -ao" and then use arrow keys to sort by the "Disk Write" column, then scroll up and down in a page in Firefox or some other browser.

Although a dissenting voice, from someone on reddit 10/2022:
Resource-monitoring software such as Task Manager on Windows and most of the resource monitors available on Linux do not make a distinction of I/O reads/writes from actual disk reads/writes. Again, I/O reads/writes ARE NOT disk reads/writes. I/O reads/writes encompass Disk, sockets, interprocess communication, among other data-transferring methods available on the OS. Firefox uses a lot of interprocess communication due to its multiprocess nature.

... Process Explorer on Windows does not show Disk reads/writes unless you run it with elevated privileges and actually enable the Disk Read Bytes and Disk Write Bytes columns. ...
But some people report that SMART data shows tons of data written.

PSD seems to interfere with a Snap-packaged browser, when the browser updates itself. I ended up disabling PSD when using Snap-packaged browser.


To fix, copy the active FF profile into tmpfs, and sync it back to disk:

# https://old.reddit.com/r/firefox/comments/pv5exw/solved_crazy_amounts_of_writes_to_ssd_by_firefox/
# https://wiki.archlinux.org/title/Profile-sync-daemon
# https://wiki.archlinux.org/title/Firefox/Profile_on_RAM
# https://github.com/graysky2/profile-sync-daemon

sudo dnf install profile-sync-daemon
psd

# Add support for a couple more browsers:
sudo cp /usr/share/psd/browsers/chromium /usr/share/psd/browsers/ungoogled-chromium
sudo cp /usr/share/psd/browsers/chromium /usr/share/psd/browsers/BraveSoftware

# I haven't tried this, but probably you could do same
# for Thunderbird profiles, by copying
# /usr/share/psd/browsers/firefox and editing to make
# /usr/share/psd/browsers/thunderbird

# Following would change it to cache just Firefox:
# Edit $HOME/.config/psd/psd.conf
# Change line:
    #BROWSERS=()
# to:
    BROWSERS=(firefox)

# See what the sync-daemon will do:
psd p

# Quit all browsers.
# Make a backup of your browser profile(s).

# Start the daemon:
systemctl --user enable psd.service
systemctl --user start psd.service
systemctl --user status psd.service
psd p
ls -l $HOME/.mozilla/firefox

# Launch browser(s) again.
# Then check disk writes:
sudo iotop -ao
Default is to sync from tmpfs to disk 1/hour. To change, create file $HOME/.config/systemd/user/psd-resync.timer.d/frequency.conf :

[Unit]
Description=Timer for Profile-sync-daemon

[Timer]
OnUnitActiveSec=
OnUnitActiveSec=10m

Have your backup scripts stop the psd service, if you're backing up browser profiles.

If you try to start the PSD service while you have any browser open, PSD will give error message and refuse to start. If you try to stop PSD service while a browser is running, the browser will be killed and then PSD service will stop.

Doesn't work for Tor Browser / Flatpak version. Browser refuses to start, says can't find profile.

For folders that are not browser profiles, similar daemon is https://wiki.archlinux.org/title/Anything-sync-daemon

Tapajyoti Bose's "Deep dive into How Web Browsers work"
Jan Kammerath's "How Browsers Work"
MDN's "Populating the page: how browsers work"

j3s's "recover lost text by coredumping firefox"

Adding JS code to a web page you have displayed in your browser: Open the Javascript console (right click page / Inspect Element) and paste in the added code.

Email Client

+/-
Formats for locally-stored email:
Maildir (each message in separate file).
mbox (multiple messages in one file; dovecot mbox).
On Ubuntu*, it looks like local "mail" is using mbox format. "ls -l /var/mail"

Message-Transfer Agents (MTAs): Postfix, exim4.
Other possible MTA's include sendmail/proofpoint, qmail, exim, dma, dmd, nullmailer ?
Is movemail an MTA that only operates between local mailboxes ?
https://wiki.archlinux.org/title/Msmtp ???

Mail Delivery Agent (MDA) is responsible for actually storing the message to disk.
Dovecot
uw-imap (on ArchWiki; but dead since 2007 or so ?)
mailutils-imap4d

From Dovecot Wiki:
"As an IMAP and POP3 server, Dovecot provides a way for Mail User Agents (MUAs) to access their mail. ... Note that Dovecot is NOT responsible for receiving mail from other servers. Dovecot only handles e-mail (a) messages coming out of the local message store, going out to IMAP and POP3 clients, and (b) messages which have already been received by the MTA and are to be stored into the local message store."

From exim Wiki:
"... things that you won't get with Exim: No POP, No IMAP. Exim is not a mailstore. It does not support IMAP or POP protocols, though it can deliver messages to mailstores that do ..."

Protocols: POP3 (read msgs from external server), IMAP (access msgs on external server), SMTP (send msgs to external server).

A client is a Mail User Agent (MUA).

Andriy Zapisotskyi's "The A-Z Of Email Development And Sending"
Email explained from first principles

  • Getting Linux local CLI mail working:
    +/- This will use the standard CLI "mail" command, and the postfix MTA.
    Other possible CLI commands (MUAs) include sendmail, mutt, maildir-utils / mu, mailx.

    [THIS SECTION IS A MESS; HAD THINGS WORKING ON KUBUNTU, THEN THUNDERBIRD DROPPED FEATURES AND I CHANGED TO FEDORA AND EVERYTHING STOPPED WORKING.]

    s I think this is just using postfix to do aliases, transferring mail locally (or was movemail doing that ?):
    
    # https://help.ubuntu.com/lts/serverguide/postfix.html
    # https://docs.fedoraproject.org/en-US/fedora/rawhide/system-administrators-guide/servers/Mail_Servers/#s3-email-mta-postfix-conf
    
    # see if postfix is installed
    apt list | grep ^postfix/
    apt show postfix
    less /etc/postfix/main.cf      # shouldn't need to change anything
    sudo dpkg-reconfigure postfix
    # if not installed:
    sudo apt install postfix
    
    sudo apt install mailutils
    
    # see if postfix is running
    sudo service postfix status
    # Or:
    sudo systemctl status postfix --full --lines 1000
    # If not running:
    sudo service postfix restart
    # Or:
    sudo systemctl enable postfix
    sudo systemctl start postfix
    cat /var/log/mail.log
    
    # forward all mail to root account to user1 account
    sudo cat /etc/aliases
    # sudo edit /etc/aliases to:
    postmaster:    root
    root:   user1
    # then
    sudo newaliases
    # then
    sudo touch /root/.forward
    # and sudo edit /root/.forward to contain
    user1@localhost
    # then
    sudo adduser user1 mail
    # Or:
    sudo usermod --append -G mail user1
    sudo service postfix restart
    sudo systemctl restart postfix
    # log out and back in
    
    # test when logged in as user1
    # note: mailbox won't exist until you send first message to it
    mail -s "subject1" root </etc/group
    mail -s "subject2" user1 <<< 'body of the message'
    mail
    

    Use postfix to send CLI mail to an external SMTP server (FAILED):
    
    # postfix: set as "internet"
    sudo dpkg-reconfigure postfix
    
    less /etc/postfix/main.cf
    less /usr/share/postfix/main.cf.dist
    less /etc/postfix/master.cf
    
    cat /var/log/mail.log
    # I can see it trying to send to external server, but failing
    
    man postfix-tls
    man 5 postconf
    man 5 master
    
    # In /etc/postfix/master.cf, un-comment line:
    #submission inet n - n - - smtpd
    # and un-comment following -o lines,
    # EXCEPT for the lines containing "restrictions=$mua_",
    # leave those commented-out.
    
    https://serverfault.com/questions/1046542/how-to-enable-tls-with-port-587-with-secure-authentication-on-postfix
    https://wiki.archlinux.org/title/postfix
    
    # Got to point where mail was going out to SMTP server, and
    # being rejected because sender was not using a FQDN.
    # Support for SMTP server says: don't use SMTP, you'll
    # be fighting the spam filters.
    

    Use mailutils-imap4d to provide an IMAP interface to local CLI mail (mbox) of current user only:
    
    sudo apt install mailutils mailutils-imap4d mailutils-mda mailutils-doc
    
    sudo systemctl status mailutils-imap4d --full --lines 1000
    # FAILED
    
    man imap4d
    imap4d --config-help | less
    
    ports TCP 143, TCP 220, UDP 220, TCP 993
    https://www.tutorialspoint.com/unix_commands/imapd.htm
    

    Use dovecot to provide an IMAP interface to local CLI mail (mbox):
    
    sudo apt install dovecot-core dovecot-imapd
    sudo systemctl status dovecot --full --lines 1000
    
    # Edit /etc/dovecot/conf.d/10-mail.conf to say:
    mail_location=mbox:~/Mail:INBOX=/var/mail/%u
    
    /etc/init.d/dovecot restart
    
    # test that local mail works; may have to install Postfix
    
    # Set client to use port 993.
    # FAILED on Thunderbird; it wants a FQDN on email address
    
    cat /var/log/dovecot.log
    
    https://www.linux.com/news/how-build-local-imap-server/
    https://www.redhat.com/sysadmin/install-configure-dovecot
    https://www.dovecot.org/
    https://docs.fedoraproject.org/en-US/fedora/rawhide/system-administrators-guide/servers/Mail_Servers/
    



  • green check-mark  Thunderbird:

    See "Thunderbird" section of my Secure Communication page



Password Manager

+/- See "Password Manager" section of my "Authentication" page.

VPN

+/-
  • green check-mark  Windscribe:
    +/- Installed Windscribe VPN client (beta) by going to https://windscribe.com/guides/linux#how-to and choosing Ubuntu and doing the commands. It worked, doileak.com finds no leaks. CLI only, no GUI app.

    I created file Windscribe.sh in my home directory, put "windscribe login" and "windscribe connect" in it, made a launcher on desktop that runs "bash /home/USERNAME/Windscribe.sh", works.

    There is no icon in the system tray, or any other indication that Windscribe is running. Have to run "windscribe status" in CLI to find out. Also "systemctl status windscribe.service".

    Windscribe modifies: iptables, ip6tables, /etc/hosts, kernel IP routing table (netstat -r), network devices list (ip -c addr), DNS list (systemd-resolve --status or journalctl -u systemd-resolved -f or resolvectl status).

    I didn't try strongSwan and IKEv2 to Windscribe:
    https://old.reddit.com/r/Windscribe/comments/6zzn0h/guide_using_ikev2_on_linux/



  • Proton VPN:

    Installed CLI and UDP and OpenVPN version, had to create OpenVPN/IKEv2 credentials on their site.

    Doesn't modify iptables like Windscribe does; I don't see any new rule with "tun0" or "tun+" in it.

    https://protonvpn.com/support/linux-vpn-tool/
    sudo protonvpn-cli -connect

    I didn't try strongSwan and IKEv2 to Proton VPN:
    https://protonvpn.com/support/linux-ikev2-protonvpn/


  • WireGuard protocol:

    WireGuard
    It's not in the Linux kernel yet, as of 5.3, I think. And even when it is available (in 5.6 ?), it won't be the full stack you need for a commercial individual-to-public-internet VPN. It will be what you need for "all traffic on LAN1 gets tunnelled to LAN2", I think.


  • VPN Look-Out Applet:

    Cinnamon-only and Proton-VPN-only, I think, as of 12/2018.

    VPN Look-Out Applet


See "VPN" section of my "Connection Security and Privacy" page.

GUI Text Editor

+/-
  • Xed:

    Default editor on Mint.

  • Kate:

    Part of KDE project.
    Multiple languages, extensions.
    Refuses to run as root/sudo.

    Using as an IDE:
    No built-in "prettier", or plug-in to do it ?
    Very weak notion of "projects" ?

    Kate
    KDE Applications' "Kate"
    KDE Store's "Kate Themes and Extensions"
    KDE TechBase's "Development/Tutorials/Kate/KatePluginTutorial"


  • Gedit:

    Default editor on Ubuntu GNOME.
    IMO the "find" functionality is very bad, I don't like it.
    Doesn't have "sort all lines".
    Does have spell-checker.


  • gnome-text-editor:

    New default editor on Ubuntu 22.10.


  • green check-mark  Mousepad:

    Default editor on Xfce, I think.
    Doesn't have "sort all lines".


  • FeatherPad:



  • green check-mark  Pluma:

    Default editor on Ubuntu MATE.
    Doesn't have "sort all lines".
    Does have spell-checker.


To make some CLI commands launch your favorite editor instead of nano or something, set e.g. "export VISUAL=/usr/bin/xed" in your .profile.

CLI Text Editor

+/-

Source Code Editor

+/-
  • green check-mark  Visual Studio Code (VS Code):
    +/-
    VS Code is a source code editor, not a full IDE like other Visual Studio editions. Which means it calls extensions and plug-ins and helper applications to do various things.

    Versions:
    +/-
    • Code-OSS: from Microsoft; base open-source code.
    • VS Code: from Microsoft; Code-OSS plus proprietary stuff; proprietary extension store; telemetry.
    • green check-mark  VSCodium: from community; Code-OSS plus open-source stuff; open but smaller extension store; no telemetry.
    • vscode.dev: from Microsoft; runs in browser. vscode.dev
    • GitHub Codespaces: from Microsoft; runs VS Code in VMs; enterprise only.
    • Code Server: from Microsoft; run VS Code in server and you access it through browser.

    VS Code is built from Code-OSS plus proprietary pieces.
    VSCodium is built from Code-OSS plus some open-source pieces.
    "Remote development extensions may have some proprietary code that's not in VSCodium."

    Rob O'Leary's "VS Code or VSCodium?"
    Rob O'Leary's "Migrating from VS Code to VSCodium on Linux painlessly"

    Another variable: packaging. Native or Flatpak or Snap ? I've had some issues with the Flatpak and Snap versions, related to sending link to browser, or talking to extension-debugger application. Maybe fixed now, I don't know.

    Another variable: install native package from your distro's repo (likely to be older), or from developer's site (newest) ?
    Linuxize's "How to Install Visual Studio Code on Ubuntu 20.04" (from MS site)

    Two stores for extensions:
    Microsoft Visual Studio Marketplace
    Open VSX Registry
    Also you could install an extension in a VSIX file from a developer's site.

    Settings
    +/-
    Easiest way to go to Settings is ctrl+comma.

    Stay in "User" tab unless you have a good reason to set a workspace-specific setting.

    Disable "minimap".
    Disable "quick suggestions" (editor.quickSuggestions = false).
    Change Emmet to exclude it from working on HTML. Or disable Emmet completely via View / Extensions.
    Disable "hover", or set time to large.
    Disable Editor: Selection Highlight.
    HTML: Format: Wrap Line Length set to 0.
    Editor: Match Brackets: off. Files: hotExit: off.

    By default, opening a file via single-click is just a "preview", and will be replaced by next file opened. To turn off this behavior, go to File / Preferences / Settings and uncheck "Workbench > Editor : Enable Preview".

    Possible: add "files.exclude":{ "**/*.jpg": true, "**/*.gif": true }

    datagubbe's "Sane Visual Studio Code"
    Rob O'Leary's "VS Code - What's the deal with the telemetry?"

    Extensions
    +/-
    Not sure what this thing called "Emmet" does; found "Emmet is a markup language for expanding CSS rules into HTML" in devhints.io's "Emmet cheatsheet"
    Chris Coyier's "Editing HTML Like A Boss In VS Code"

    "HTMLHint" by ctf0 (diagnoses HTML syntax errors).
    "CSS Peek" by Pranay Prakash (adds Peek sub-menu when right-click on CSS in HTML file).

    Can't find an HTML link-checker extension for VS Code. Ended up building my own: "HTML / XML / RSS link checker" by Bill Dietrich

    Candidates:
    Code Spell Checker by Street Side Software (got annoying, left it installed but disabled)
    Web Accessibility by Max van der Schee.
    VS Code Printing Free by PD Consulting.
    Markdownlint by David Anson.

    To see what key-shortcuts map to what extensions: ctrl+k ctrl+s then search on "@source:extension".

    Rob O'Leary's "VS Code - You don't need that extension"

    Once you have a lot of files open, Ctrl+p to get a drop-down that makes it easy to bring a file to the front.

    VS Code seems to be auto-saving to temp files in the background, which has saved me a number of times when my system froze due to hardware problems.

    To see all key-shortcuts: ctrl+k ctrl+s.

    Workspaces
    +/-
    Was struggling to get VS Code to remember and re-open files when quitting/relaunching and hopping between projects. Found the solution: do not do ANYTHING with Workspaces, just use "Open Folder" and "Save All" and "Close Folder". It remembers open editors on a folder basis, and opening/creating a workspace destroys that somehow. Don't use Workspaces at all.

    From /u/digicow on reddit 4/2020:
    +/-
    In the simplest sense, a workspace is a domain of VS Code settings. The VS Code settings domains are hierarchical, starting with User, then Remote (if you're connected to a remote server using VS Code Remote Development extensions) per-server, then Workspace, then Folder.

    So at any level, you can override a VS Code setting from a higher level, with a more-specific one. This allows you to have settings that just apply to a folder, or across an entire server, etc.

    While the other layers are "natural" (they exist, regardless -- every file is in a Folder, so you can set Settings for that folder), Workspaces are "intentional". By creating a specific workspace, you now have the ability to set Settings for it separate from those other layers.

    Normally when you use VS Code, you open it up with a particular directory (e.g., a project). If you make Folder-specific settings in that directory, they'll be stored in a .vscode sub-directory.

    But with Workspaces, you can have multiple directories, which might be in completely unrelated places in your filesystem, as "multiple roots" of the VS Code window. Perhaps 2 different, but related projects, or an application project and a related library project. The associated settings for this are stored in a workspace file, which you can save anywhere.

    From someone else on reddit:
    +/-
    > What is a VS Code workspace?

    A workspace is an open folder, or a bunch of them, all in your VS Code window.

    For example, you open a folder from your terminal with the code command. That folder is now in the root of your workspace. Then you open another folder with the menu: File / Add Folder to Workspace... Now you have two folders open that are nowhere near each other on your hard drive, but they exist in the same workspace.

    If you're going to need those folders open pretty often, you can do: File / Save Workspace As... and give it a name. You can also set configuration options just for that workspace, and do things like save code snippets just for that workspace, and set debugging workflows just for that workspace.


    Remote development, from someone else on reddit:
    +/-
    When you're connected to a remote project, everything happens on the remote machine. Building and running happens on the remote machine. When you open a terminal, you're logged in on the remote. When you use "open file", you are presented the remote file system (with different UI because it doesn't use the native file dialog).

    Bug reports: use Help / Report Issue menu item in VS Code app.
    reddit's /r/vscode/
    reddit's /r/vsCodium/
    VS Code extensions
    VS Code "Native Crash Issues" (how to report VS Code crashes)
    VSCodium (binaries minus MS telemetry; maybe can't get extensions from Marketplace)

    Burke Holland and Sarah Drasner's "VS Code can do that ?"
    Pandiyan Murugan's "VS Code useful Tips and Tricks"
    Espen's "Productivity in Visual Studio Code"
    Fireship's "25 VS Code Productivity Tips and Speed Hacks" (video)
    Antoine Mesnil's "Code faster with custom VS Code snippets"
    VS Code's "Emmet in Visual Studio Code"

    Matthew MacDonald's "Setting up JavaScript Debugging in Visual Studio Code"
    Matthew MacDonald's "Basic Debugging in Visual Studio Code"
    Shai Almog's "16 Missing Features in the VS Code Debugger"

    Sebastian Andil's "Tips to use VS Code more efficiently"
    sandeepk27's "VS Code Shortcuts for Web Developers" (sequences you type that expand)
    Hariprasath's "10 Tricks Every Developer Should Know in Visual Studio Code"
    Saurabh Palatkar's "VS Code Extensions For Near IDE Experience"
    viatsko/awesome-vscode
    codeSTACKr's "Learn Git in 30 Minutes" (video) (git and VS Code)
    Angelo's "How to install Code Server on Ubuntu 22.04"

    See the Building a VS Code Extension section of my Develop an Application page.

    Use git from inside VS Code:
    +/-
    I have a project on disk with git already set up and working to connect to GitHub, using CLI commands. To connect to it through VS Code:
    1. Close current folder/project in VS Code (ctrl+k f).
    2. In the left-side panel, click icon to open the Source Control tab.
    3. Click "Open Folder".
    4. Open the project folder.
    5. See files in the folder. Those NOT included in the repo will have "U" (Untracked) next to the name.
    6. Edit a file.
    7. ???
    https://www.digitalocean.com/community/tutorials/how-to-use-git-integration-in-visual-studio-code




  • Eclipse Theia:
    +/-
    Open-source equivalent to VS Code, but split into front and back ends, and back end can run in the cloud.
    Theia
    Theia 1.0 release announcement
    Supports use of VS Code extensions.
    Open VSX Registry



  • Atom:
    +/-
    Discontinued 12/2022. But community build.

    Install HtmlHint, and once it gets running, any file that has an error will have its name underlined in red in the project pane. Try putting "<<" in a file to test it.

    But Atom is a pain. Slow to launch. No easy way to open just HTML files in a folder. No easy way to move from one tab/pane to another. Always opens Project window that shows all files in folder, not just open files. Flatpak version doesn't support commands such as "atom" in CLI. Ended up uninstalling that version, and installing directly from atom.io site. Now "atom" command at CLI works.

    Flatpak version, with supporting framework, takes almost 6 GB of disk space !

    Later, I read "Open Atom, then open command palette (Ctrl+Shift+p). Type "shell" and select "Window: Install Shell Commands" from the drop-down suggestions."

    reddit's /r/Atom/
    Atom Packages

    Building a new package for Atom:
    +/-
    Nick Tikhonov's "Building your first Atom plugin"
    Jeremy Heleine's "How To Develop a Package for GitHub's Atom Code Editor"
    Atom's "Hacking Atom"
    Vincent Composieux's "Create your first Atom package"
    Atom's "atom-ide-ui (A collection of Atom UIs to support language services)"
    Atom IDE
    Atom's "bottom-dock package"
    Package to copy ? lettertwo / atom-narrow-provider-diagnostics

    Getting started:
    1. Install the non-Flatpak version of Atom, from the atom.io web site.
    2. On CLI, do "apm -v" to verify that apm is okay.
    3. On CLI, do "atom -v" to verify that atom CLI app is okay.
    4. In Atom, Ctrl+Shift+P to open Command Palette, then find "Package Generator: Generate Package" and run it.
    5. Type name/path of new package. It will be created and a new instance of Atom opened on it.
    6. For any Node.js packages you need, go to project directory in CLI and do "npm install --save PACKAGENAME" and "apm install". Package should appear in an entry in package.json. And node_modules directory tree will be created and/or populated.
    7. To build a package that uses IDE/language features, do "apm install atom-ide-ui".

    While developing:
    1. Change code.
    2. Open the Command Palette (Ctrl+Shift+P) and run the "Window: Reload" command.
    3. Navigate to Packages > YOURPACKAGENAME and do whatever commands it implements.
    4. Repeat.

    I tried to port my HTML link-checker extension from VS Code to Atom, and gave up. Couldn't understand basics, and there's no standard Diagnostics pane in Atom, and Facebook is dropping support of some key stuff they developed for Atom.




  • Pulsar:

    Pulsar


  • Zed:

    Zed
    SavvyNik's "Zed the VSCode Killer" (video)


  • KDevelop:

    KDevelop
    Wikipedia's "KDevelop"
    Runs on Linux, Mac, Windows.
    Looks like plug-ins are written in C++.


  • IntelliJ:

    Wikipedia's "IntelliJ IDEA"

    HTML support only in Ultimate edition.


  • Webstorm, from JetBrains:

    Chris Bongers' article


  • Micro:

    micro


  • Brackets:

    Brackets
    Multiple languages, extensions.


  • Notepadqq:

  • Notepad Next:



  • Geany:

    Geany


  • Sublime Text:

    Costs $80.
    Sublime Text


  • Vim:
    +/-
    Been using vim for 2 years now, mostly because can't figure out how to exit
    vim-tiny (the compact edition) may be installed by default. You probably want to remove it and install full vim (which includes vimtutor). I also installed Vim-addon-manager.

    Type "vi" on CLI to run it.
    Run "vimtutor" to get a tutorial.
    Run "vim-addons" to manage addons (scripts).

    Vim
    devhints.io's "Vim scripting cheatsheet"
    reddit's /r/vim

    Vim has same UI as the old Unix "vi", as far as I can tell. To get out, ":q" or ":wq".

    how to exit cvim
    vi = original from 1970's, lacks syntax-highlighting and plug-ins.
    vim = from 1991, has syntax-highlighting and plug-ins, bad scripting language Vimscript.
    neovim = from 2015, is vim plus scripting language Lua.


Fireship's "I tried 10 code editors" (video)
meain's "What is in a modern code editor?"


GitHub:
Alistair Ross's "Howto: What is Git and Github? How do I use it and why should I care?"
See my "Git and GitHub" page

Markdown editors:
+/-
Add Markdown AllInOne + Markdown Preview Enhanced extensions to VS Code. Or:
Boostnote
Remarkable
voldyman / MarkMyWords

Microsoft's "Learn Markdown reference"
adam-p's "Markdown Cheatsheet"


PDF Viewer and Editor

+/-
Adobe no longer supports Linux for PDF viewing and editing ? But article.

I think "annotating" a PDF is not the same as "doing form-filling". Neither is same as "editing". And there are two types of form-filling: "XFA" and "AcroForms". Then there is "signing".

Some PDF's contain an internal loader or something: they show "wait for file contents to load, if it doesn't load you need new PDF software", and then the real contents should appear. Mostly that kind of PDF doesn't work on Linux, in my experience.

  • Evince (AKA Mint's "Document Viewer" AKA xreader):

    "sudo apt install evince"
    Snap image fails to launch, throws error.
    Works fine for simple PDF files, but does not work for form-filling (XFA or AcroForms ?) documents (many tax forms).


  • Foxit Reader:

    Didn't work on my tax PDF; I think doesn't support form-filling.
    Foxit Reader


  • Master PDF Editor:

    "flatpak install net.codeindustry.MasterPDFEditor"
    Supports form-filling in free version, but very bad support, at least on the file I tried in 1/2021 and again 1/2022. In PDF file, check-box field values aren't saved, pull-down menus don't work, help buttons don't work, more. Free version adds a watermark to saved file ? CAN display much of XFA form file correctly, so not totally useless. In 3/2022, version 5.8.3 worked better, but still form buttons don't work.
    Master PDF Editor


  • PDF Studio ($99 but there is a free trial):

    PDF Studio


  • PDF Studio Viewer:

    PDF Studio Viewer


  • Okular:

    Opened a PDF that has XFA form fields, and Okular (1.11.2, 1/2021) said "XFA forms are unsupported".
    Has a "show forms" option somewhere ?


  • Scribus:

    "sudo apt install scribus"
    Can export to several image formats.


  • Firefox browser:

    Can view PDF and do form-filling. But form buttons may not work, network access won't work.


  • Microsoft Edge browser:

  • Acrobat Reader DC:

    "snap install acrordrdc"
    Installs a snap of a deprecated version of WINE (!), gives scary warnings, I stopped the install.


  • green check-mark  Ampare PDF To Image:

    "snap install amparepdftoimage"
    Very plain GUI app to view PDF and save page to PNG format. Works.


  • pdftk:

    CLI only.
    "snap install pdftk" or "sudo apt install pdftk".
    Doesn't convert to other formats.
    Especially good for combining PDFs into one bigger doc.

    From someone on reddit:
    I was not looking forward to today because I was under the impression that I was going to have to set up a cracked version of Windows on a VM so that I could use a free trial of Acrobat Pro to do a simple (but time-sensitive) PDF edit on my xubuntu machine.

    That was before I stumbled upon pdftk.

    The process was quite simple:
    
    sudo apt install pdftk
    pdftk PDF_File.pdf burst # strip out all pages and creates individual .pdf files
    
    #Sign the signature page with GIMP
    #Create a new page to add new text with LibreOffice 
    #Paste signature into new page and export as PDF
    
    ls *.pdf >> pdf-filenames.txt #Create file of individual page names
    value=$(<pdf-filenames2.txt) #Assign pagenames to bash variable
    pdftk $value cat output Merged_Document.pdf #Merge the files back into one
    
    [There is also Python package to do stuff like this, PyPDF2.]


I ended up having to go to a Windows machine to do my PDF tax forms.

Edit PDF in LibreOffice Draw then export to PDF
poppler-utils (and "man pdfinfo") ? PDF Chain / pdfchain ?
quickfill add-on for Chromium ? There are other extensions in Chrome Store.
Xournal (does annotation, writing over top of a PDF that is used as the background image).

Useful online service: Pdf2Jpg.net
https://www.pdfescape.com/open/ (doesn't display XFA forms)
https://pdfbob.com/editor/ (doesn't display XFA forms)

qpdfview ?
mupdf ?

Angelo's "How to extract embedded images from a PDF"

Diagram And Flowchart Editors

+/- Ankush Das's "Top 10 Microsoft Visio Alternatives for Linux"
Sidrah Abdullah's "Linux Network Diagram Tools"
Comparisons between Text to Diagram tools

Farm-Fresh web icons

Produce diagrams that use ASCII characters:
ASCIIFlow
dot-to-ascii

Genealogy (family tree)

+/-
  • Gramps:

    Gramps
    Create a person, then click on Relationships tab, and add related people.
    Easier than adding all the people, then trying to link them together.


  • GeneoTree:

    GeneoTree


  • GeneWeb:

    GeneWeb


  • GenealogyJ:

    GenealogyJ


Steve Emms' "8 Best Free Linux Family History Software"

Web Site Tools

+/-
  • Sitemap generator:
    +/-
    I need to generate an HTML site-map for my web site.

    It's a hand-crafted site, so I can't just use a Wordpress plug-in or something. The directory structure is not the same as the hierarchy you get from following links, so I can't just use output from "ls" or "find".

    Did internet searches for products/projects. Found several that were dead links. Several that wanted to run as PHP on my web server (don't want to do that).

    xmlsitemapgenerator.org won't handle a page bigger than 80KB (many of my pages).

    Gave up on generating HTML directly, will settle for XML. xml-sitemaps.com gave lousy output, no hierarchy, listed links to all kinds of non-HTML files. Same with npmjs.com/package/sitemap-generator-cli. patrickmn.com/projects/osg gives some kind of HTTPS protocol version mismatch.

    Decided to reorganize my web site so the directory structure matches the logical organization, at least at the top level, and then I can use a "find" command to generate most of the site-map. Then changed my mind, decided to go the opposite direction, flatten the directory structure and hand-craft the site-map.



  • Link-checker:
    +/-
    Looked at wget, Klinkstatus, LinkLint.

    Found LinkChecker. Installed Linkchecker-gui; won't check external links by default, you have to uncomment "#checkextern=1" in config file. Works okay. But often it reports the same bad link in a file 2 or 10 times.

    Ended up building my own VS Code extension to do link-checking: linkcheckerhtml

    Page not found



  • tidy:

    CLI HTML syntax-checker and prettifier.

    
    sudo zypper install tidy
    
    # None of the following modify your HTML files:
    
    tidy -errors -quiet FILENAME.html >/dev/null
    tidy -errors -quiet --mute-id yes FILENAME.html >/dev/null
    
    tidy -errors -quiet --show-filename yes --fix-uri no --mute UNKNOWN_ENTITY,CONTENT_AFTER_BODY,REMOVED_HTML5,ILLEGAL_URI_CODEPOINT,ESCAPED_ILLEGAL_URI *.html >/dev/null 2>report.txt
    
    for f in *.html; do echo "$f" '---------------------'; tidy -errors -quiet "$f" >/dev/null; done
    

    Gives a fair number of false-positives, incorrectly reports parameters in URLs, doesn't understand nested lists.


  • HTMLHint:

    HTML syntax-checker used by Atom, VS Code, other editors.

    Official site
    Project home
    HTMLHint can be called from CLI and JavaScript and maybe other languages.

    Connector from HTMLHint to VS Code: Microsoft / vscode-htmlhint


  • Open-source tool to generate static HTML site: Hugo

    I tried using Hugo, and it was a fiasco. See "Tried Hugo" section of my "Your Personal Web Site" web page.

Alistair Ross's "How to password protect web sites via .htaccess"
Alistair Ross's "Quick and dirty hacks: one line HTTP Server"

Downloading Videos and Images

+/-
VLC has a "Record" function that's supposed to let you save any video VLC is playing, but Record totally sucks, don't use it.

For downloading videos, use browser add-on "Video DownloadHelper" by mig, and install the companion app that it uses.

Or, invoke the companion app directly on CLI:

youtube-dl -f mp4 -o OUTFILENAME.mp4 https://www.youtube.com/watch?v=SOMETHING
# or:
youtube-dl -x --audio-format mp3 -o OUTFILENAME.mp3 https://www.youtube.com/watch?v=SOMETHING
Or use "yt-dlp" instead of "youtube-dl"

Give playlist URL to yt-dlp, download all videos in playlist.
Tube Converter (front-end for yt-dlp)

Viewing streaming TV channels: Hypnotix.
angeloma's "Install Hypnotix - The Linux Mint IPTV client"
Divine's "Hypnotix: A New IPTV Streaming App for Watching Live TV, Movies and Series"
Additional providers:
https://iptv-org.github.io/iptv/index.m3u


Streaming a video from computer to smart TV

+/-
Some hardware ways to do it: HDMI cable, SD card.


MiniDLNA / ReadyMedia / ReadyDLNA package:
Computer acts as a media server.
"Player" app on TV or smartphone looks for media server.

https://sourceforge.net/projects/minidlna/
https://wiki.archlinux.org/title/ReadyMedia

"sudo systemctl status minidlna --full --lines 1000"
"cat /etc/minidlna.conf"
"man minidlnad"
"top -p `pgrep minidlnad`"
If service doesn't work, try "sudo minidlnad -v -d" on CLI.

In GUFW, have to turn off "deny incoming", or add a rule "allow-in-TCP-port8200".
In firewalld, have to add "minidlna", "ssdp", "upnp" to "public" and "home" ?
Have to turn off VPN on computer.
Computer can be on either Ethernet or Wi-Fi.
Have to allow outbound port 1900 (PnP) to minidlna on computer.
Make all video files have a+r permissions.

Test by accessing from app "DLNA Movie Play" on Android smartphone.
Have to turn off VPN on smartphone.
Select server, select file to play, select TV to play it to.
App controls playback and volume etc; I assume playback dies if you quit the app.
If you restart the server, also quit out of the app and re-launch the app.
App won't play to TV if TV is playing through an app such as Netflix ?

Test by accessing from app "DMS Explorer" by OHMAE Ryosuke on Android smartphone.
App can play from DLNA server to either phone or TV.
App gives lots of connection details, but then any error gives uninformative "Error occurred" message.

Runyon Canyon's "How do I connect my DLNA server to my Samsung Smart TV?"

MiracleCast

Screen-mirroring:
Have to turn off VPN on computer.
Have to turn on Wi-Fi on computer.
"iw phy | grep P2P-client" should display a "P2P-Client".
"iw dev" should display a "P2P-device".
Install "gstreamer-plugins-vaapi"
"gnome-network-displays" (should work in any DE)
Wants to create some firewall zone, but fails, and doesn't say what zone.

Zeeman Memon's "How to Stream from Linux to Your TV"
Ankush Das's "Top 9 Best Linux Media Server Software"
UbuntuPIT's "13 Best Linux Media Server Software for Making a Home Theater PC"
Arch Wiki's "Miracast"
Miracast


Recording Desktop Activity

+/-
Screenshot to image file:
  • green check-mark  gnome-screenshot (screenshot):

    Printscreen key: screenshot of the entire screen.
    Shift+Printscreen: screenshot of a region selected by mouse.
    Alt+Printscreen: screenshot of current window.
    CLI: gnome-screenshot --help

  • In Firefox (screenshot):

    Ctrl+shift+s

    Also, if you want a screenshot of something that doesn't fit on the visible screen (requires scrolling): do ctrl-shift-I, go to Inspector, find that thing's "node" in the DOM, right-click on it, select "Screenshot Node".

  • Flameshot (screenshot):

    Flameshot
    flameshot-org / flameshot
    Writwik Ray's "Install, Set Up, and Use Flameshot in Linux"

    UI confusing at first. Launch Flameshot, icon appears in system tray, click on it to start a screenshot. Get purple dialog that says "use mouse to select an area, or Enter for whole screen". But Enter copies whole screen to clipboard; instead use Ctrl+S to copy whole screen to file. If you use mouse to select an area, you can then do things, including Ctrl+S to save to file. Only saves in PNG format.

  • spectacle (screenshot; KDE):

  • swappy (screenshot; Wayland):

Screen-recording to GIF or video file:
VokoscreenNG
Linux Uprising's "Display Keystrokes And Mouse Clicks In Screencasts Using KmCaster"

Recording CLI Activity

+/-

Image Viewing and Editing

+/-
View only:
+/-
  • KDE's Pix:

  • feh:

    feh
    CLI-oriented.

  • Ristretto:


Simple editor (at least flip/rotate, crop, resize):
+/-
  • green check-mark  X-Apps Pix:

    Installed by default in Linux Mint. It's a fork of gThumb.
    Editing features include flip/rotate, crop, resize, more.
    Has plug-ins.
    Has file-rename (F2).
    Can't get it to start in full-screen mode; have to hit F11.
    Open-With context menu item so you can open image in a more-complex editing app.
    linuxmint / pix
    Some improvements I'd like to see: setting to open straight into full-screen mode, setting for Delete key mapped to Delete or Move to Trash menu item or nothing, Edit / Preferences / Extensions / "More Extensions..." just takes you to GitHub page for the whole app, turning off "General - Ask confirmation before delete" setting doesn't work (or requires restart ?).
    If in View mode files seem to be presented in a strange order, go to View / Sort By and change order.
    To prevent opening multiple instances of the app, go to Edit / Preferences / General and check "Reuse the active window to open files".
    To prevent extra step in every Save As, in lower-left corner of Save As dialog un-check "Show Format Options".

    You can add a script, by going to Hamburger icon pull-down and selecting Personalize. Filename in command is represented by "%F". Numeric key-pad shortcut-keys work only if NumLk is on.

    Install in systems other than Mint*:
    auroralinux's "The Guides to Install Linux Mint's X-Apps on Ubuntu"
    Go to http://packages.linuxmint.com/list.php?release=Ulyana and download and install in this order: xapps-common, pix-data, pix.
    You'll have to update them manually, every few months maybe.

    Settings for Pix are stored in gsettings; use dconf-editor to see org.x.pix tree; also "gsettings list-recursively org.x.pix".

  • green check-mark  Gwenview:

    Default in KDE.
    No way to do single-instance-only.
    Sort-order of files doesn't match that in Dolphin ?
    No way to increase contrast.

  • Eye of GNOME (eog):

  • Eye of MATE:

    No resize or crop.

  • Xviewer:

    Installed by default in Linux Mint.
    The only editing features are flip/rotate.

  • XnViewMP:

    It lives in /opt/xnview.
    Its mostly a slide-show app; editing features are flip/rotate, crop. No resize.

  • IrfanView:

    No native version for Linux; have to use a Windows emulator such as WINE.

  • gThumb:

    Slide-show plus; editing features are flip/rotate, crop, resize, draw, color/focus.
    Has plug-ins.
    For me, it failed the most basic UI test of a viewer: arrow-keys don't work to move to previous/next image.
    X-Apps Pix was developed by starting with the gThumb codebase.

  • nomacs:

    "flatpak install org.nomacs.ImageLounge"
    Not actively developed ?

  • Mirage:

    Not actively developed ?

  • ShowFOTO:

    Medium-level image editor.
    Didn't see any good way to view previous/next image in folder.

  • Nemo-image-converter extension to Nemo:




Complex editor:
+/-
  • Krita:



  • green check-mark  Pinta:

    Image editor and drawing.
    Seems to do everything I was using GIMP to do.
    But the text-tool is very weak.
    And trying to install add-ons fails; version mismatch problem.


  • Lazpaint:

  • GIMP:

    Heavy-duty image editor. It "saves" to its native format xcf, and "exports" to something else such as jpg.

    The only features I use/need:
    - expand canvas size (add more blank area around image without changing size of image)
    - edit image that has a transparent background layer
    - select all pixels of a certain color, change to another color

    From someone on reddit:
    "In the main menu under 'Window' there is an option for 'single window mode'. That will help make GIMP more like Photoshop. Move and place the panes where you like them. I also use GIMP Paint Studio which is an addon with more brushes, palettes, patterns, gradients, tool presets and menus. There are many addon and plugins for GIMP."

  • Drawing:

    New default app in Mint 19.3, supposedly replaces GIMP.
    Aadesh's "Drawing - A basic Image editor"

  • DigiKam:

  • kImageAnnotator:

    Mark up images by adding text, arrows, other items.
    ksnip / kImageAnnotator
    Open any image with ksnip and it's just kImageAnnotator.

  • Annotator:

    Mark up images by adding text, redactions, other items.

  • ksnip:

    Mark up images by adding text, redactions, other items.
    Dedoimedo's "Ksnip - Make my screenshots pretty please"

KolourPaint, phatch, Photoscope, Shotwell, photopea, photoflare.

I briefly tried Inkscape, and found it very complex. Couldn't figure out how to draw arrows, no built-in shapes, etc.

CLI image manipulation:
+/-
  • ImageMagick:

    "sudo apt install imagemagick"
    "man ImageMagick"
    https://imagemagick.org/script/download.php

    Useful commands:
    
    # Debian-style rename command; apparently Red Hat version is different.
    rename 's/ //g' *
    rename 's/.mp4.png/.png/' *.png
    for i in *.png ; do j=`basename "$i" .png` ; echo "$j" ; convert "$i" -quality 92 "$j".jpg ; done
    rename 's/png.jpg/.jpg/' *.jpg
    
    identify -verbose FILENAME
    
    identify -format %w FILENAME
    echo $W
    
    convert -resize "${MAXWIDTH}" OLDFILENAME NEWFILENAME
    
    # Add 1-pixel black border to image:
    convert -bordercolor black -border 1 OLDFILENAME NEWFILENAME
    


  • exiftool:
    
    sudo apt install libimage-exiftool-perl
    
    # display EXIF data using very short tag-names
    exiftool -veryShort FILENAME
    
    # remove all EXIF data from files
    exiftool -all= -progress FILENAMES
    rm *original
    # or
    exiftool -all= -progress FILENAMES | grep Error
    rm *original
    # or
    for i in *.jpg; do echo "Processing $i"; exiftool -all= "$i"; done
    
    # remove geotag EXIF data from files
    exiftool -geotag= -progress FILENAMES
    # maybe
    exiftool -gps:all= -progress FILENAMES
    


  • exiv2:
    
    # remove all EXIF data from files
    exiv2 -q rm FILENAMES
    


  • imgp:

    FOSS Linux's "How to Resize Images by Command line in Ubuntu"


  • mogrify:


  • artem:

    Bobby Borisov's "How to Convert Images to ASCII Art"


File-renamer: F2

Find duplicate images: rdfind.

Online tools:
+/-
Change background color of a PNG to transparent: LunaPic

Kendra D. Mitchell's "How to Make GIF Background Transparent and Change Color" (OIE worked for me)
EraseBG (I haven't tried it)
rembg (I haven't tried it)

Add a border around an image: Super Tool

Side-by-side images: SideIt

Change contrast: InstaEditor

Diagram-editor: diagrams.net (formerly draw.io)

Serious editor: Photopea


Video Player

+/-
  • green check-mark  Celluloid:

    Installed by default in Linux Mint. Formerly GNOME MPV.

    Editing features include:
    "s" to take a screenshot from the video (best, I think),
    "ctrl+s" to take a screenshot of what appears in the window,
    "p" to pause,
    left/right arrow to go back/forward 5 seconds,
    shift+left/right arrow to go back/forward 1 second,
    ","/"." to go back/forward 1 frame.


  • VLC:

    munix.dk's "VLC configuration for Ubuntu"


  • SMPlayer:

    Set preferences to use MPV or MPlayer engine.


  • Dragon Player:

    Installed by default in Fedora KDE.


  • mpv:

Linux Uprising's "How To Enable Hardware Accelerated Video Decode In Browsers"

Video Editor

+/-
Everyone was harmed during the making of this video

For serious recording and streaming, the gold standard seems to be OBS.

Tried Kdenlive and Openshot-qt video editors, but way too complicated for me, all I want to do is cut segments out of existing videos.

For just cutting:
+/-
  • ffmpeg:
    
    ffmpeg -i INPUT.mp4 -c copy -ss [starting time in hh:mm:ss] -to [finish, in hh:mm:ss] OUTPUT.mp4
    
    [If you have NVIDIA hardware, ffmpeg can be recompiled to use GPU instead of CPU.]

  • Installed VidCutter. Unfortunately a couple of GB of stuff came with it; it uses KDE stuff and flatpak. But the app does what I want, without too much hassle. I think it's using ffmpeg on the back-end.

  • LosslessCut


Openshot
Olive
Kdenlive.
Shotcut.
Flowblade (article).
I've heard: avoid Davinci Resolve; installation and distro support are horrible, and it's NVIDIA-oriented (crashes a bit on AMD ?).
Pitivi.
Lightworks.

FOSS Linux's "How to capture screenshot GIF, and Video with Audio, from command line"
Alistair Ross's "Screencast recording with Green Recorder"
Rotating videos with FFmpeg
SK's "20+ FFmpeg Commands For Beginners"
SoByte's "FFmpeg Video Processing Tutorial for Beginners"
MP4 to GIF: "ffmpeg -i FILE.mp4 -vf fps=10,scale=450:-1 FILE.gif"

Audio

+/-
The gold standard for audio handling/editing is Audacity. But every operation rewrites the audio to disk, so be careful when editing, and expect heavy load on disk.

Check MP3 file:

sudo apt install libimage-exiftool-perl
exiftool -veryShort FILENAME
exiftool -veryShort -x IngredientsFilePath -x IngredientsToPart -x IngredientsFromPart -x IngredientsDocumentID -x IngredientsMaskMarkers -x IngredientsInstanceID -x Lyrics -x Lyrics-eng -x Comment -x Comment-eng -x Composer -x Album -x Artist -x Band -x Subtitle -x HistoryAction -x HistoryInstanceID -x HistoryWhen -x HistorySoftwareAgent FILENAME

# part of ffmpeg package
ffprobe FILENAME

# http://www.ibiblio.org/mp3info/
sudo apt install mp3info
mp3info -x -f -F FILENAME

# https://sourceforge.net/projects/mp3diags/
# http://mp3diags.sourceforge.net/
sudo apt install mp3diags
mp3diags

# Extract audio from video file:
ffprobe INPUT.mp4 2>&1 | grep 'Audio:'
ffmpeg -i INPUT.mp4 -map 0:1 -c copy OUTPUT.aac
ffmpeg -i INPUT.mp4 -vn -acodec copy OUTPUT.mp3

SoX Sound Exchange: Klaatu's "Convert audio files with this versatile Linux command"
sox also has aliases "play" and "rec", can do playing and recording.

Best music-player: Lollypop ?

Play audio from CLI:
If ffmpeg is installed, "ffplay FILENAME.mp3".
If VLC is installed, "nvlc FILENAME.mp3".
Or install "mpg123" and do "mpg123 FILENAME.mp3".
"qdbus org.kde.kglobalaccel /component/kmix invokeShortcut 'increase_volume'

MIDI files have a fairly complex binary format.
Outline of the Standard MIDI File Structure
StudioCode.dev's "MIDI note numbers and Middle C"
"od -c /usr/lib/all_notes_off.mid"
"sudo pw-cat --midi --playback /usr/lib/all_notes_off.mid"
"man amidi"
"man aplaymidi"

Jay Link's "Simple Sounds for Linux"
hilbix / kdmktone

See "Fixing Audio" section of my "Linux troubleshooting" page.

Encryption etc

+/-
Encryption of data at rest: see "Encryption of data at rest" section of my "Linux Storage" page

Encryption of data in motion:
+/-
  • green check-mark  Tor Browser:

    Installed Tor Browser by downloading an archive and extracting from it. But trying to put it in a location shared by all users (/usr/local/bin) caused a permission nightmare and it wouldn't run. Put it in my home directory and it works, but I don't see it in the GUI menu of applications.

    See Tor Browser section of my Connection Security page



See the "Messaging" section of My "Secure Communication" page.

See the "Linux Software" section of my "Backups" page.

See the "Linux" section of my "Anti-Malware" page.

For info about iptables, firewalls, Firejail, Apparmor, SELinux, and more, see my Linux Network and Security Controls page.

CLI Shell

+/-
Bash and zsh, and maybe sh and dash, are POSIX-compliant; fish is not. So there are scripting constructs that will work on bash and zsh but not on fish.

  • Bash:

    LeCoupa / bash-cheatsheet.sh
    Greg's (GreyCat's) Wiki
    bash-it
    SK's "How To Customize Bash Prompt In Linux"
    Mendel Cooper's "Advanced Bash-Scripting Guide"
    Jeff Kaufman's "You Should Be Logging Shell History"

    Useful things to set in $HOME/.bashrc:
    
    HISTSIZE=2000
    
    # https://www.gnu.org/software/bash/manual/bash.html#index-HISTIGNORE
    HISTIGNORE='reboot.*;shutdown.*'
    
    # When shell exits, append to history file instead of overwriting it
    shopt -s histappend
    

    Useful interactive commands:
    
    # At an empty command-line:
    sudo !!   # run previous command with sudo in front
    Up-arrow  # reuse previous commands
    Ctrl+r    # search command history
    history | grep PATTERN    # see all times you used PATTERN
    for i in $(cat FILENAME.txt); do CMDNAME $i; done
    
    # In middle of a command:
    # First there is command-line editing:
    Alt+.     # reuse previous command arguments
    Alt+B     # move left 1 word
    Alt+F     # move right 1 word (but collides with File menu)
    # See "man readline".
    # See "less /etc/inputrc", override in "$HOME/.inputrc".
    # Then there is command completion:
    Tab       # filename completion
    # See "less /usr/share/bash-completion/bash_completion".
    # Add more in /etc/bash_completion.d/
    # See active definitions: "complete".
    



  • Zsh:

    If you want to use ZSH:
    1. Check to see if it's installed already: try running command "zsh".
    2. If not found, install through Software Manager, or on CLI do "sudo apt install zsh" and "sudo apt install zsh-completions".
    3. Also install oh-my-zsh
    4. Do "sudo usermod -s /bin/zsh [user name]".
    5. Log out of that user and log in again ?
    6. Maybe copy some lines from "$HOME/.bashrc" to "$HOME/.zshrc" (aliases), and from "$HOME/.bash_profile" to "$HOME/.zprofile" (code that starts the X Window System) ?

    Awesome-zsh
    Neal Parikh's "Customizing your shell prompt"
    Anthony Heddings' "What is ZSH, and Why Should You Use It Instead of Bash?"

    I used zsh and oh-my-zsh for a while and found I was using none of their features, couldn't be bothered to learn them, too many other things I wanted to do instead. Went back to bash.

    To uninstall later:
    1. "$HOME/.oh-my-zsh/tools/uninstall.sh"
    2. Do "sudo usermod -s /bin/bash [user name]".
    3. Log out of that user and log in again ?



  • Fish:

    fish
    Scripting syntax differs from that of bash.


There are lots of other alternatives: dash, ksh, oksh, csh, tcsh, loksh, mksh, yash, etc.

Useful things to set in $HOME/.profile (affect all shells):

export PAGER="most"

Terminal/shell shortcuts: Sakib Hossain's "Useful Linux Terminal Shortcuts"

What I want in a terminal emulator: ability to launch maximized, setting for font type and size, simple. Okay: xfce4-terminal.

Ctrl+r history-search on steroids: McFly.

See "Linux Shell Script" section of my "Develop an Application" page.

Useful CLI snippets

+/-

# Delete 4th line from a file, changing file:
sed -i '4d' FILENAME

# Delete blank lines from a file, changing file:
sed -i '/^$/d' FILENAME

# Delete lines that start with a certain word, changing file:
sed -i '/^WORD/d' FILENAME

# Edit strings, changing file:
sed -i 's/OLD/NEW/' FILENAME

# In lines containing STR, edit strings, changing file:
sed -i '/STR/s/OLD/NEW/' FILENAME

# Change block of text across newline, not changing file:
sed '$!N;s/STARTPATT\nENDPATT/NEWTEXT1\nNEWTEXT2/;P;D' FILENAME

# Get lines that do NOT match pattern, and output to stdout:
grep -v 'PATTERN' FILENAME

# Process a group of whole lines, and output to stdout:
awk '/STARTPATTERN/,/ENDPATTERN/ { print }' FILENAME
awk '/STARTPATTERN/,/ENDPATTERN/ { gsub(/PATTERN/,"NEW"); print }' FILENAME

# Get value out of a file full of "KEY=VALUE" lines:
( . /etc/os-release && printf '%s\n' "$NAME" )

# Make lots of files:
seq 1 100 | xargs --verbose touch

# Basic shell loop:
for file in *; do wc -l "$file"; done
while true; do echo 1; sleep 5; done      # infinite loop

# Find any HTML file that does not contain an h1 tag:
grep -L '<h1' *.html

# To dump a log file into a pastebin:
wgetpaste log.txt

# Open a file using GUI app:
xdg-open FILENAME        # uses default app
mimeopen -a FILENAME     # asks what app to use

# See if a file contains any unicode chars:
diff -u FILENAME <(cat -v FILENAME)

Useful for converting among various file formats such as CSV and JSON: miller.

For JSON also the jq, jo, and fx commands. Or install package yajl or yajl-tools.

Vivek Gite's "How to use htmlq to extract content from HTML files"
mgdm / htmlq (extracting from HTML code)
HTML to text: "lynx -dump -stdin" "lynx -dump URL" "w3m -dump URL" also htmlq

Linux Shell Tips' "How to Parse or View XML Code in Linux Command Line"
"yq" command can translate between XML and other formats.
xidel: article, home
To install: download tar.gz file, extract contents, cd into it, sudo bash, ./install.sh, creates /usr/bin/xidel.
https://stackoverflow.com/questions/26484140/how-to-sort-xml-file-in-unix
"xidel --xquery '<root>{for $i in doc("file.xml")/*/* order by $i/pid, $i, $i/date return $i}</root>' --output-format xml"
"xidel --xml --xquery '<opml version="1.0">{for $i in doc("feedlist.opml")/*/*/* order by $i/body/outline/text return $i}</opml>' >out.opml"
Couldn't get it to work !!!

Kai Yuan's "Find Processes by Keyword"

To find the CLI (file) name of a GUI application:
- look in app's Help/About menu item.
or:
- click on Start button, go to app's entry in the start menu, right-click on it, select "edit application".
or:
- look for app's .desktop file in /usr/share/applications or $HOME/.local/share/applications, and get filename from "Exec=" line in there.



Witness the power of this bash script

Office

+/-

  • LibreOffice:
    +/-
    Easy Linux tips project's "LibreOffice: configure it right"
    See item "Libre Office: improving Macro Security" in Easy Linux tips project's "Security in Linux Mint: an explanation and some tips"
    Easy Linux tips project's "Problems with Libre Office? Install a newer Libre Office"
    Dedoimedo's "LibreOffice 7.0"
    Dedoimedo's "LibreOffice 7.1"
    Dedoimedo's "LibreOffice 7.2"
    Dedoimedo's "LibreOffice 7.3"
    Dedoimedo's "The biggest blocker to LibreOffice adoption? LibreOffice."
    Supports newer XML-formats: .docx, .xlsx, etc.

    My installation got stuck in some loop where, upon launch, it would say "recovering file" or something, then fail and say it again, forever. Uninstalled all of LO. Had to do it one component at a time; just uninstalling the meta-package didn't do it. And there were about 15 components. Rebooted, and installed just Libreoffice-writer and Libreoffice-impress.

    From someone on reddit 7/2019:
    [For small-business use:]
    Libreoffice is a great piece of software if you will be delivering PDFs as the final output, but if you will be sending a Word or Excel document back and forth, you should use WPS Office, Microsoft Office 365 Web Apps, or Google Docs.

    From someone on reddit 9/2020:
    LibreOffice can save some documents that are CLOSE to Office format. Macros, scripts, templates all get fracked up. Many Uni's OUTRIGHT say it's MS Office or you don't get credit for ANY submitted work. Check with your Uni.

    From someone on reddit 9/2020:
    Libreoffice is trash for anyone who needs to do a lot of work in spreadsheets.

    One very easy example is eliminating duplicates. In Excel, you click the column letter, and then click "remove duplicates", and you're done. This is very common in spreadsheet work.

    In Calc, you do a special search, and then you have to select the entire range you want to search (so if I have 1500 rows with some blanks, I have to click A1, then CTRL+downarrow a bunch of times), then you select something to HIDE duplicates, then you copy the visible rows to a new sheet or to some area that is not filtered, and then you delete the old data. That's f*cking absurd for a function that's done ALL the f*cking time in spreadsheets.

    MS Office understands why people use these applications, and it puts all the most commonly used things right there on the Home tab. LibreOffice vs O365 is like me vs Usain Bolt in a 100M sprint. It's so bad that I've considered going back to Windows for my workstation, but I'm generally not responsible for creating or maintaining spreadsheets right now, so Excel online is workable.

    Install "mscorefonts".
    "LibreOffice macros and MS macros are partially compatible".

    Business-model: Some problems


  • Microsoft Office 365:

    But then you're sharing your documents with Microsoft.
    Microsoft Office 365
    Excel macros not supported ?


  • OnlyOffice:

    OnlyOffice
    Doesn't support newer XML-formats: .docx, .xlsx, etc.


  • WPS Office:

    WPS Office for Linux
    I'm told the UI is a fairly straight copy of MS Office's UI, so a user would be comfortable moving between the two.
    Best document compatibility with real MS Office.

  • FreeOffice:

    FreeOffice


  • Abiword:



  • Google's "Office Editing for Docs, Sheets & Slides":

    Extension for Chrome browser.
    Installed it into Chromium browser, it installed okay but didn't work.


Word Online: can be used for free by anyone with an Outlook.com or Hotmail account.

Office 365 is a home and business subscription service. Some subscription plans offer desktop Office but others don't. Some plans include web services like business email and Azure AD but others don't.

From someone on reddit 7/2019:
[For small-business use:]
Honestly, Linux Desktop isn't really business-ready at the moment. It's getting close but it's not there.

For office work you need Microsoft Office, be it Word, Powerpoint, Visio, or Excel.

Some will say you can use LibreOffice, or other open-source. But the main problems are the same tasks are not always possible (try setting sequential formulas in an excel sheet with the Ctrl + Enter on Libre), and when you create documents in these alternatives, they don't look the same when opened with Microsoft Office (i.e. vendors or clients you deal with will see this as unprofessional).

From someone on reddit 8/2020:
I have hundreds of existing PPTs which I use at work, and unfortunately these don't show well on Mint or any other Linux distro I've tried, whether on Libre, or any other Office replacement. The layout is all wrong, elements spill out of the slides, so much so that the slides are unusable without spending hours rearranging everything, so I stick to Win10 for work.

The same issue happens on Office online, with the additional problem that it doesn't have all the features of the desktop apps.

If you need to create new files, Libre etc. are a good alternative, but if you need to use existing files, unfortunately MS is still the way to go.

An issue I see, as a home user, is that the printer drivers don't properly shrink an oversize document. Maybe it's a bug in my distro (Mint 19) or apps or the driver for my printer (HP 363x series), or because I'm in Europe using A4 paper and an A4-sized printer. But a document with content right up to the edges, and slight bigger than the paper size, has the edges cut off instead of being shrunk to fit. Happens with xed and xreader at least. And there're no margin settings in their print dialogs. [Edit: not a problem with same printer with Ubuntu 20.04 and gedit app.]

Mohammed Abubakar's "Microsoft Office On Linux Is Now A Thing, Thanks To WinApps"

Robert Zak's "How to Open a docx File without Microsoft Office"
Convert Word documents to Clean HTML
pandoc (convert among document formats)

For info about Docker, flatpak, appimage, and more, see my VMs and Containers/Bundles page.

Database

+/-
See what you have installed already; don't remove it:
"apt list | grep -i -E 'maria|mysql|postgres|sqlite|redis|mongodb' | grep installed"


Remote Access (Remote Desktop) to Linux machine

+/-
Heard on a podcast, don't know if true:
Some remote control (x2go) requires an open port for SSH, some (TeamViewer) doesn't.
X-based (x2go) is fast, VNC slow.

Protocols: RFB/VNC (ports 5500, 5900), RDP (port 3389), NX (uses SSH ?), XDMCP (port 177), SPICE (???), EXEC (???), SSH (port 22), TeamViewer (various; can HTTP tunnel ?).


Recommended: X2Go (uses SSH, so secure from the start; article)

TeamViewer (installs a version of Wine, daemon runs as root, not a lot of features).

AnyDesk

RustDesk

VNC ? ("sudo apt install x11vnc", then in GUI run x11vnc, accept incoming connections and apply, "sudo netstat -tulp | grep vnc", on Windows machine install VNC Viewer, run it, connect and login, get full GUI desktop access)

noVNC ?

Remmina ?

Mehedi Hasan's "Fast and Secure Remote Desktop Clients for Linux"
Bobby Borisov's "Remote Desktop with Linux"
Seth Kenlon's "How to connect to a remote computer using VNC in Linux"
Wikipedia's "Comparison of remote desktop software"

Remote Access (SSH) to another Linux machine

+/-
[ALL OF THIS COULD BE WRONG]

Single-session SSH clients:
+/-
OpenSSH, PuTTY, Butterfly, kitty, more.

You could run one of these multiplexing session managers, then run a single-session SSH client in one of their sessions:
OpenSSH client info is in /etc/ssh/ssh_config file.
OpenSSH server info is under /etc/ssh/ssh_config.d directory.
Any time you change server config, do "sudo systemctl restart sshd.service"
Create a new SSH key-pair (and password for it) for this user, on client: "ssh-keygen -t rsa"
Copy public key for this user to login as USERNAME to server: "ssh-copy-id USERNAME@SERVERADDR"
To login as USERNAME to server: "ssh USERNAME@SERVERADDR", get asked for password of key-pair on client.

Games

+/-
Ankush Das's "Top 31 Best Linux Games You Can Play for FREE"
Nick Congleton's "The 10 Best Free Linux Games"
Steve Emms' "42 of the Best Free Linux Games"
Derrik Diener's "5 best free Linux FPS shooters to checkout"

System76's "The System76 Guide to Gaming on Pop!_OS"
Dedoimedo's "Steam Play, Proton - How be things in 2021?"
You can run Steam in a sandbox: "firejail steam".


Dota 2:
Install Steam first, from Software Manager.
VPN not allowed while installing.
Dota 2 installs 24 GB under $HOME/.steam
Try to run game, get "Unable to start game - Failed to create an OpenGL context - Your graphics card must support at least OpenGL v3.1".
I'm using Mint 19.3, kernel 5.3.0-46-generic, Intel integrated i915, X.Org 1.19.6, "OpenGL: renderer: Mesa DRI Intel Ironlake Mobile v: 2.1 Mesa 19.2.8".
Set Dota 2 launch options in Steam to include "LIBGL_ALWAYS_SOFTWARE=1": Launch Steam, right-click on Dota 2, click on Properties, click on Set Launch Options, set field to "LIBGL_ALWAYS_SOFTWARE=1 %command%".
ArchWiki's "Steam"
While running, Steam and Dota 2 together will take more than 1.5 GB of RAM.
Performance is atrocious, unplayable on my slow laptop with no special GPU.

E-Book / epub Readers

+/-
green check-mark   Foliate: set Advanced / Continuous mode.
green check-mark   Okular.
Calibre: insists on maintaining a library, has bugs in cross-page navigation.
coolreader: no way to go up/down one line on page.
fbreader: no way to change text size.
Bookworm: doesn't support epub format ?
Koodo Reader
epy (CLI)



Finding apps:
AlternativeTo
ArchWiki's "List of applications"
AppImageHub
One Thing Well blog
harrytran103 / awesome-linux-apps



What I want in a file-explorer: all folders same view, run file in terminal, handle LUKS containers, two tabs in side-by-side mode, start maximized.
Okay: Dolphin (KDE) ?
Not: Thunar (Xfce). YNNNY. But features are being added.
Not: doublecmd. YNNYY. Columns not good. article
Caja: not really all folders same view, does have run file in terminal.
Nemo ?
Mehedi Hasan's "Linux File Manager: 20 Reviewed"



Some applications are written to work only in a specific GUI framework, such as KDE or GNOME. Others are written to work inside a cross-platform framework, such as Electron or Node.js or Ruby Rails, that then has versions which run inside various lower frameworks, such as KDE or GNOME.



There are some application-deployment frameworks, such as Docker and Ansible (Ansible: article1, article2, article3, article4 ).



Note: CLI tools "apt" and "apt-get / apt-clone / apt-config" give identical functionality but differ in emphasis:
"apt" is intended for interactive use, and may change slightly over time.
The others are intended for scripting / back-end use, and try to stay constant over time.
Someone said the two groups use separate caches.



Easy Linux tips project's "Firefox: optimize its settings"
Easy Linux tips project's "Google Chrome and Chromium: improve their settings"

Alistair Ross's "Review: Download Managers for Linux"

App Outlet

pkgs.org (finding and downloading newest versions of packages)
Repology

Lilite: A Linux Autoinstaller

Fonts:
ArchLinux's "Font configuration"
"apt-cache search ^fonts-" and "apt search ^fonts-"
Install fontconfig-infinality.
cryzed / fix-infinality.md

Munif Tanjim's "How To Install Google Earth on Ubuntu Linux"
But it's available through Software Manager too.
If all searches go to equator, edit /opt/google/earth/free/googleearth (or /opt/google/earth/pro/googleearth ?) to add a line "export LC_NUMERIC=en_US.UTF-8" before line that starts with "LD_LIBRARY_PATH".

cboxdoerfer / fsearch (fast file search utility)
Joey Sneddon's "Linux File Search Tool 'Catfish' Just Got Even Faster"

Check hash of a file you downloaded:
Alexandru Andrei's "How to Verify Authenticity of Linux Software with Digital Signatures"
drewblay / Compare-File-To-Hash



US taxes:
FOSS Post's "Open Source Tax Software For Filing US Taxes"
OpenTaxSolver



My "Develop a Desktop Application" page





Things To Do

Clippy want me to help you with Linux ?
Work your way through some basic tutorials:
Linux Journey
LinuxCommand.org
Linux Survival
Ubuntu's "Using The Terminal"
Linux command line for you and me
Ryans Tutorials' "Linux Tutorial"
Julia's Drawings

Chris Hoffman's "The Linux Directory Structure, Explained"
[At CLI do "man hier" (canonical hierarchy) and "man file-hierarchy" (extensions in systemd systems ?).]
Debian's "Device Names in Linux"

Far more in-depth:
Sven Vermeulen's "Linux Sea" [BROKEN LINK]
David A Rusling's "The Linux Kernel" (circa 1999)
The Linux Kernel documentation



For info about tightening and testing security and privacy, and more, see my Linux Network and Security Controls page.



Updating

+/-
Terms:
  • Update: install new versions of packages as they come out, maybe daily.

  • Upgrade to new point-release: e.g. from 19.2 to 19.3. Usually happens same way as daily updates ? Usually works fine.

  • Upgrade to new major release: e.g. from 20 to 21. Usually some special mechanism ? More dangerous.

  • Fresh install (AKA "nuke and pave"): back up your data and custom changes, then completely wipe the disk and install new OS, install apps, then restore your data and settings of some major apps.

  • Fresh install of OS only: you can try this if you have / and /home in different partitions. Wipe and install OS and apps on /, hope that /home remains intact and works. I don't do it.


Updates:
On Mint, run Update Manager application. Application and kernel updates will appear automatically. If you want to change kernel "lines" (e.g. change from 4.15.x to 5.0.y), use View / Linux Kernels menu item.

On Ubuntu, to manually update kernel, use Ukuu (Ubuntu kernel update utility) or bkw777 / mainline. But the normal update app automatically will update kernel and remove old kernels. To see kernels, do "dpkg --list | grep linux-image". To purge several of the oldest images, do "sudo apt purge IMG1 IMG2 IMG3".

On Ubuntu, to see when snaps were last updated, do "ls -l /snap".

For Ubuntu, I asked about installing a LTS release and then later updating to a non-LTS release, and got this from people on reddit:
[From LTS to non-LTS,] You don't need to do a fresh install. Ubuntu will update with its software update tool. If your first install was an Ubuntu LTS, you will need to configure Ubuntu to allow non-LTS upgrades though. If you open "software sources" and go to the "Updates tab" configure "Notify me of a new Ubuntu version" from "Long term support" to "any new version". When the time comes you'll be prompted to update.

Word of advice: Avoid using third party software repositories (like PPAs) as they can cause situations where Ubuntu can't do an OS update. This is the number 1 cause for Ubuntu failing to upgrade. I personally don't have any as most of the stuff I use is available through Ubuntu's software library.

...

In major updates, the updater tool will disable your PPAs so that there are no conflicts.


Upgrades:
New point-release upgrades seem to go okay, usually.

Major release upgrades: I hear that Ubuntu and its derivatives usually do these okay. On other distros, maybe not so much.

Fresh install:
Instead of doing a major release upgrade, I do a fresh install. And a fresh install shows that your backups are complete, and of course is necessary when switching to a different distro.




Special hardware

+/-
  • WD My Passport encrypted external hard disk drive.

    How to install and use KenMacD's "wdpassport-utils":
    +/-
    1. sudo apt install git
      (might already be installed, by default)
    2. git clone https://github.com/KenMacD/wdpassport-utils $HOME/wdpassport-utils
    3. Now you have the files in $HOME/wdpassport-utils
    4. You could move the files to /usr/local/bin so they can be used all users. But this particular software is going to hard-code a password into a file, so best to leave it under your home.
    5. cd $HOME/wdpassport-utils

    6. Follow the "Non Gui Steps" from the README.md file in the repository:

      First time:
      1. sudo apt install sg3-utils

      Every time:
      1. Plug in external drive and wait a few seconds.
      2. sudo dmesg | grep sg | grep "type 13"
        This should return one line which gives drive's "sgN" (such as "/dev/sg1").
      3. ./cookpw.py YOURPASSWORD >password.bin
      4. Verify that size of password.bin is 40 bytes.
      5. sudo sg_raw -s 40 -i password.bin /dev/sgN c1 e1 00 00 00 00 00 00 28 00
        but replace "sgN" with appropriate number, "sg1" or whatever.
        See message saying something like "Good".
      6. Now drive should be unlocked.
      7. sudo partprobe
      8. Now drive should be mounted and available for use.
      9. Delete password.bin file so password is not saved ?

    7. Try the GUI method:

      First time:
      1. Instructions say do "sudo apt install gksu", but that package has been removed from Debian/Ubuntu18/etc.
      2. Edit gui.py to change all "gksudo" to "sudo".
      3. sudo apt install python-qt4
        May take 30 minutes or so in Live session.

      Every time:
      1. python gui.py
      2. See buttons for unlocking and mounting the drive.
      3. App automatically makes a desktop icon gui.py, called "WD Unlocker", which you can use next time.
      4. It creates a password.bin file each time. Delete that file after drive is accessible, or after unmounting disk ?

    8. To remove the drive from the system:
      1. In File Explorer, right-click on drive and click "Safely Remove Drive".
      2. Wait 5 seconds or so.
      3. In File Explorer, right-click on drive, see that now there is a "Mount" option (showing drive is unmounted), and click "Safely Remove Drive" again.
      4. In File Explorer, see that the drive no longer appears.
      5. In File Explorer, right-click on WD Unlocker drive, and click "Safely Remove Drive".
      6. Unplug the external drive.


    Use WD-Decrypte with sg3-utils. See Vector's "Unlock WD Password Protected HDD in Linux" (video) which explains how to use WD-Decrypte, covering a couple of glitches not explained in Sifo Hamlaoui's "HOW TO DECRYPTE Western Digital DRIVE ON LINUX" (video).

    Or use 0-duke's "wdpassport-utils"

    Or use KenMacD's "wdpassport-utils"

    How to install KenMacD's "wdpassport-utils" (or other GitHub project), from someone on reddit 8/2018:
    1. sudo apt install git
      (might already be installed, by default)
    2. git clone https://github.com/KenMacD/wdpassport-utils $HOME/wdpassport-utils
    3. Now you have the files in $HOME/wdpassport-utils
    4. You could move the files to /usr/local/bin so they can be used all users. But this particular software is going to hard-code a password into a file, so best to leave it under your home.
    5. The rest of the steps are specific to this project.
    Installing a package using apt installs it system-wide by default (which is why you have to use "sudo").

    A better long-term solution:
    1. Mount the encrypted drive on a Windows machine (or Linux, if you have decryption working).
    2. Copy the files to a temp disk.
    3. Disable hardware encryption on the encrypted drive (must be on Windows to do this).
    4. Plug the encrypted drive into your Linux machine.
    5. Reformat it as a VeraCrypt volume with ext4 or NTFS or Btrfs filesystem inside.
    6. Copy all the files from temp disk to VeraCrypt drive.
    7. Close VeraCrypt drive, re-open, close, unmount disk, re-mount, test several times.
    8. Save backup copy of VeraCrypt volume header on some other device.
    9. Erase temp disk.






After using Linux for a while

+/-



/r/linuxmint
Easy Linux tips project's "Complete starters' guide for Linux Mint"
Linux Mint
community.linuxmint.com's "The Linux Mint User Guide"
community.linuxmint.com's "Tutorials"





Miscellaneous



Gary Newell's "Complete List of Linux Mint 18 Keyboard Shortcuts for Cinnamon"

OSTechNix's "3 Good Alternatives To Man Pages Every Linux User Should Know"
TLDR pages ("simplify the beloved man pages with practical examples")
Sudeshna Sur's "Replace man pages with Tealdeer"
ManKier ("a collection of man pages, translated ... to semantic HTML5")
"curl cheat.sh/COMMANDNAME"

A bit old, but interesting: Debian Reference's "Chapter 9. System tips"
GNU Project's "All GNU packages"
LeCoupa / bash-cheatsheet.sh
P. Lutus's "How to Use Secure Shell"
KernelNewbies
OverAPI
Debian Reference
The Debian Administrator's Handbook
The Linux Foundation's "Introduction to Linux (LFS101)" course



CLI stuff

+/-
"man": in output, /SEARCHTERM to find, n for next match, Shift+n for previous match.
"man -k KEYWORD": list all man pages that relate to KEYWORD.
"apropos KEYWORD": list all man pages that relate to KEYWORD.
"info" (mainly for GNU components).
"whatis COMMAND": show brief description of command.

Type command and space-tab-tab, see all possible arguments to it.

Type chars and tab-tab, see all commands that start with those chars.

Try "exa" (article) instead of "ls"; it adds coloring and tree features.

jlevy's "The Art of Command Line"



Keyboard

+/-
To change keyboard behavior/mappings, see man pages for setxkbmap and xkeyboard-config. Or on Mint, go to System Settings app, Hardware / Keyboard / Layouts, select a layout [probably English (US)], click Options, click on Caps Lock behavior.

How to type non-English keys on an English keyboard:
Set Compose key to something (I use "Pause"), for typing non-English characters.
Seth Kenlon's "Use secret keyboard keys on Linux"
In KDE: Settings / Keyboard / Advanced / Configure keyboard options / Position of Compose key / Pause
Fsymbols' "Linux keyboard shortcuts for text symbols"
ArchWiki's "Key combinations"
€ == compose = e
€ == compose C =
¿ == compose ? ?
ñ == compose ˜ n

ArchWiki's "Keyboard backlight"
Slimbook-Team / slimbookrgbkeyboard

See the "Keyboard" section of my "Linux Troubleshooting" page.



Fonts

+/-
Run the "Fonts" or "Font Manager" GUI application.

In GNOME, setting system fonts is done through gnome-tweaks, not the Fonts app. But it doesn't affect what is shown in fc-match on CLI; confused.


Font sets you could add:
+/-
Microsoft windows font pack ?
Later followed instructions in SK's "Install Microsoft Windows Fonts In Ubuntu 18.04 LTS":

sudo apt update
sudo apt install ttf-mscorefonts-installer
sudo fc-cache -f -v		# build font cache files

Maybe Windows Vista fonts too ?
See ZcomTech's "How to install Microsoft fonts in Linux office suites 2020":

sudo apt install fontforge
wget https://gist.github.com/maxwelleite/10774746/raw/ttf-vista-fonts-installer.sh
chmod a+x ttf-vista-fonts-installer.sh
sudo bash -c ./ttf-vista-fonts-installer.sh
rm ttf-vista-fonts-installer.sh

Some people say the Ubuntu fonts are superior to most, also the Noto fonts are good. Download an archive and extract into /usr/share/fonts. Then go to settings and change system font.

On Fedora, "sudo dnf copr enable dawid/better_fonts" then "sudo dnf install fontconfig-enhanced-defaults fontconfig-font-replacements" then log out/in.

Pop!_OS fonts: "sudo dnf install 'mozilla-fira*' 'google-roboto*' fira-code-fonts"

gsfonts ?
ttf-croscore ?
fonts-croscore ?

Braille Institute's "Atkinson Hyperlegible font" ?
Download from https://brailleinstitute.org/freefont or https://fontmeme.com/fonts/atkinson-hyperlegible-font/ , extract *.otf files, put the font files in /usr/share/fonts/opentype or $HOME/.local/share/fonts/opentype, run "fc-cache". Change system settings to use the font, or look at a web page that uses the font. "fc-list | grep -i atk" One way to tell it's worked: zeroes should now have backslashes in them. Also look in Fonts application.


If available, try changing "antialiasing" from "Standard/grayscale" to "Subpixel/LCD".

Someone said "hinting=full , subpixel order=rgb, anti aliasing=enabled, backend=glx, bytecode interpreter=true".

Fonts can look bad if monitor is set to wrong refresh rate (too low) ?

At the CLI level:

fc-list             # all fonts installed in system
fc-match serif      # font you get for "serif" use
fc-match sans-serif # font you get for "sans-serif" use
fc-match monospace  # font you get for "monospace" use

for family in serif sans-serif monospace Arial Helvetica Verdana "Times New Roman" "Courier New"; do
  echo -n "$family: "
  fc-match "$family"
done

ArchWiki's "Font configuration"
Hemant Bhandari's "How to Install a Font in Linux?"
Bobby Borisov's "How to Install Fonts on Linux"
Chuan Ji's "How To Set Default Fonts and Font Aliases on Linux"
Venam's "Did You Know Fonts Could Do All This?"
Aria Beingessner's "Text Rendering Hates You"



Printing

+/-
Types of printer interface:
  • CUPS printer drivers (which are somewhat uncommon).
    Seth Kenlon's "How to set up your printer on Linux"

  • CUPS PPDs (PostScript Printer Definitions). on Wikipedia

  • IPP (Internet Printing Protocol; AKA "driverless"). on Wikipedia
    The printer has to be up and responding to you when you add it to the CUPS server (because CUPS immediately tries to query it for its printing information).


"lpstat -a"
"sudo cupsctl"

"apt install ink" and then "ink -p usb" to see ink levels in a USB-connected printer.

Apparently HP printers have had great support for Linux for a long time, and open-source drivers. But now they're starting to require an internet connection and HP Smart account in the setup process. Maybe should start avoiding them.

Well-recommended: Brother printers. Cheapest inkjet models 2x the price of HP's, and don't include scanner. Might be best to get a separate scanner ? Inkjet cartidges not any cheaper than HP. Not sure about availability of cartridges where I live (Spain). Consider color laserjet printer.

Also recommended: Epson Ecotank. "High upfront cost but their ink is really reasonably priced."

Instead of using CUPS, you might be able to print directly to a network-accessible printer (has an IP address). Try "nc IPADDRESS 9100 <FILENAME" or "gs FILENAME | nc IPADDRESS 9100".

HP laser-printer: get it on LAN with a static IP address, install hplip, run "hp-setup". Then print a page. Also try accessing the printer through a browser to "IPADDRESS:631".

Aadesh's "Installing HP Printer Drivers on Linux"
Aadesh's "Installing Canon Printer Drivers on Linux"


Sharing a printer from Linux to Windows:
How To Create A CUPS Shared Printer (PDF)

OS-agnostic info:
InkTonerStore's "50 Common Printer Problems and How to Fix Them"
Anna Cruz's "13 Common Printer Problems and How to Fix Them"

To use Wi-Fi printer, may have to set firewall to allow port 631 (IPP) and 5353 (mDNS).



Scanning

+/-
If no application installed by default:
flatpak install org.gnome.SimpleScan



Time service

+/-
Time can be set via one of two daemons (actually, there are many more):
  • NTP:

    "sudo systemctl status ntpd --full --lines 1000"
    "cat /etc/ntp.conf"

    How to change NTP daemon settings:
    Get new values from NTP Pool Project's "How do I use pool.ntp.org?"
    Edit /etc/ntp.conf
    "sudo systemctl restart ntpd"
    "sudo systemctl status ntpd --full --lines 1000"

  • systemd-timesyncd:

    "sudo systemctl status systemd-timesyncd --full --lines 1000"
    "journalctl --since=today | grep systemd-timesyncd"
    "cat /etc/systemd/timesyncd.conf"

    How to change systemd-timesyncd daemon settings:
    Get new values from NTP Pool Project's "How do I use pool.ntp.org?"
    Edit /etc/systemd/timesyncd.conf
    "sudo systemctl restart systemd-timesyncd"
    "sudo systemctl status systemd-timesyncd --full --lines 1000"




Connect Android phone via USB cable to Linux

+/-
ArchWiki's "Media Transfer Protocol"
Lloir's "HOWTO: Linux and MTP" (old)
Wikipedia's "GVfs"


# Connect a phone, accept connection on phone screen, and then:
lsusb
lsusb -D /dev/bus/usb/BUSNUMBER/DEVICENUMBER
udevadm info /dev/bus/usb/BUSNUMBER/DEVICENUMBER
udevadm info /dev/bus/usb/BUSNUMBER/DEVICENUMBER | grep DEVLINKS
ls -l /run/user/$UID/gvfs/

man mtp-tools

systemctl status | grep mtp | grep -v grep

pamac list | grep mtp
mtpfs
mtp-detect | less

pamac info gvfs-mtp   # for GNOME/GTK file managers
pamac info kio-extras # for KDE file managers

sudo pamac install gmtp

jmtpfs --listDevices	# list MTP devices
# gmtp, gvfs, gvfsd-mtp, mtpfs ?
# https://github.com/whoozle/android-file-transfer-linux
See "USB" section

spion / adbfs-rootless



See what distro/kernel versions you are using

+/-

cat /etc/*release
cat /etc/upstream-release/lsb-release
cat /etc/debian_version
cat /proc/version
lsb_release -a

echo $XDG_SESSION_TYPE
echo $XDG_CURRENT_DESKTOP
echo $SHELL



See what CLI commands you use most often:

history | awk '{print $2}' | sort | uniq -c | sort -nr | head -n 10



See if you are using any 32-bit software

+/-

lsof | grep i386-linux-gnu && echo "Found 32-bit library in use" || echo "No 32-bit library in use"
dpkg -l | grep "^ii" | grep ":i386" && echo "Found 32-bit packages installed" || echo "No 32-bit packages installed"
Jesse Smith's "Checking for 32-bit applications on the operating system"



RAM (memory) stuff

+/-
To test RAM from OS: "sudo memtester 14G" (I have 16 GB of RAM).

See "Add memtest to GRUB" section

From someone on reddit:
The Linux kernel has a built-in memtest which you can activate with memtest=17 kernel parameter (requires CONFIG_MEMTEST=y in .config /proc/config.gz). It tests the RAM with random pattern, single bit pattern, and finally zero. and any bad RAM it finds is automatically reserved and won't be used any more.
I tried it, and after decrypting disk I got a black screen of 30+ seconds, which didn't show up in the system journal so I think it was running in GRUB before the kernel was loaded. No indication of pass or fail or bad memory locations.

Karim Buzdar's "How to Run Memtest in Ubuntu 20.04"



CPU stuff

+/-
"sudo tlp-stat --processor"
"lscpu"

See any vulnerabilities and if mitigations have been implemented:

tail -n +1 /sys/devices/system/cpu/vulnerabilities/*
lscpu
Wikipedia's "Processor Info and Feature Bits"

See if your system has any "ucode" packages installed, as in "pamac search ucode" or "dnf list --installed | grep firmware".

Non-zero if virtualization is enabled in BIOS:
"grep -c -E 'vmx|svm' /proc/cpuinfo"

Test for AMD Ryzen RDRAND bug:

wget https://cdn.arstechnica.net/wp-content/uploads/2019/10/rdrand-test.zip
unzip rdrand-test.zip
./rdrand-test/amd-rdrand-bug

CPU Performance

+/-
Phoronix says edit /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor to set to "performance", but my system doesn't have the lower part of that tree. I did:

sudo bash
sudo echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
But it made no difference, and after a reboot they had all been changed to "ondemand".

Steve Scargall's "How To Set Linux CPU Scaling Governor to Max Performance"
Linux Uprising's "Get Better Desktop Responsiveness Under Heavy CPU Load Using CFS Zen Tweaks"
Kernel.org's "amd-pstate CPU Performance Scaling Driver"
ArchWiki's "CPU frequency scaling"

Michael Larabel's "The Desktop CPU Security Mitigation Impact On Ubuntu 20.04" (10% perf loss on somewhat-older CPUs)
Bill Toulas' "VMware: 70% drop in Linux ESXi VM performance with Retbleed fixes"
Leo Chavez's "Disabling Intel and AMD CPU Vulnerability Mitigations in Debian and Ubuntu"
Linux Reviews' "HOWTO make Linux run blazing fast (again) on Intel CPUs"
See state of mitigations: "grep -H '' /sys/devices/system/cpu/vulnerabilities/*"
Edit kernel command-line to add "mitigations=off" (see Force fsck in GRUB or Force fsck in systemd-boot).
I turned off CPU mitigations, and saw no difference in benchmarks.

Didn't do: "sudo apt install cpufrequtils" and "echo 'GOVERNOR="performance"' | sudo tee /etc/default/cpufrequtils"

Did "sudo systemctl disable ondemand" and rebooted. That got rid of Phoronix's "CPU governor is on" warning message, but no change in results. Enabled service again.

In BIOS change "performance profile" from "Silent" to "Balanced".

DON'T overclock or undervolt or whatever. A recipe for errors and damage.

Tweaking scheduler priorities of processes:
"man nice"
Odysseas Kourafalos' "How to Control App Priorities with Ananicy"


Processes and Threads

+/-
It used to be (Unix / 1970s - 1990s) that processes were a kernel thing and threads were a compiler/binary thing. The kernel didn't know about threads, and all scheduling was done at the process level.

Now, the kernel knows about threads (through system calls such as pthread_create()). "man pthreads". Each thread in a process has separate program counter value, stack, register values, scheduling, signal masks, capabilities, and CPU affinity. Everything else is shared by all threads within a single process. when the process terminates (through any thread calling exit(), or other ways), all threads and the process die.

Kernel threads versus user-space threads ? The old model was user-space threads only (with cooperative switching among threads inside each process). The new model has a kernel thread associated with each user-space thread (with preemptive switching among threads), as well as kernel-only threads (all descended from kthreadd) not associated with any user-space address. Inside the kernel, each thread is represented by a task_struct.

From someone on StackOverflow:
A thread is an execution path through your program's code. That's pretty much all there is to say about that.

A process is a container for all of the resources needed by an instance of your program; The process has a virtual address space, it has open file handles, it may have sockets, it may have memory mappings, it may own IPC objects, I'm not sure what all else. And, a process has some number of threads. (Always at least one thread, sometimes more.)

The threads in your program do all the work. The process is the place in which they do the work.

From people on Lobsters.rs, about Spencer Baugh's "The Unix process API is unreliable and unsafe":
The big problem on Linux is that it conflates threads and processes in the kernel: threads are processes that share an address space and a file descriptor table. On both Linux and FreeBSD, after vfork and before exec, you can mark a process to terminate when its parent exits, which avoids it leaking. On Linux, the conflation of processes and threads means that this causes the child process to exit as soon as the thread that created it in the parent exits, making the API completely useless.

...

It's even weirder: Threads on Linux don't even have to share a file descriptor table, and processes can share a file descriptor table and even their address space. Using clone3, You can make all kinds of fun Franken-tasks that live in the twilight zone between thread and process.

OpenGenus's "Linux threads: Creation, Data passing, Waiting"
Himanshu Arora's "Introduction to Linux Threads"




Swap

+/-
"swapon --show"

  • Swap file ("ls -l /swapfile").

  • Swap partition.

  • Compressed RAM (zram; "zramctl").

  • Compressed RAM cache in front of a swap device (zswap 1).

  • No swap.

Also "free -h" and "cat /proc/swaps".

You could have no swap, or use zram, instead of a swap partition or swap-file. Doing so would reduce use of SSD.
"swapon --show"
"sudo swapoff -a"
But this is not persistent across reboot.
Comment out swap line in /etc/fstab.
Also check contents of /etc/crypttab.
Don't remove /dev/vgkubuntu-swap_1 partition; I did that and something didn't like it.

Special procedure for creating a swap-file on Btrfs: ArchWiki's "Btrfs: Swap file"


Gary Newell's "Do You Need A Swap Partition?"
Hayden James' "Linux Performance: Why You Should Almost Always Add Swap Space"

Linux Uprising's "How To Use A Swap File Instead Of A Swap Partition On Linux"
FOSS Linux's "How to create or add a SWAP partition in Ubuntu and Linux Mint" (but "method 1" creates a swap file)
Aaron Kili's "How To Create a Linux Swap File"
Ryan Sechrest's "System running out of memory: create a swap file"


You may see swap space used even when there is free RAM space: something could get swapped out when RAM is full, later does not come back in unless needed. Lengthy discussion of swap and swappiness: Chris Down's "In defence of swap: common misconceptions".
Also see Linux ate my ram ! (but see PR).

Swappiness: "default value is 60 and means that swap comes into operation when 40% of the RAM is in use."
You could change "swappiness" to 0 to reduce traffic to swap partition on SSD.
"sudo sysctl vm.swappiness"
Edit /etc/sysctl.conf and add "vm.swappiness=0"



Out-Of-Memory (OOM)

+/-
"free -h"
"top"
"htop -t"
"htop -t -F PATTERN"
"systemd-cgtop"
Amin Nahdy's "How to fix high memory usage in Linux"


OOM-handlers: kernel, nohang, oomd, earlyoom, systemd-oomd, oom_reaper.
"sudo ps -ex | grep oom"
"cat /etc/systemd/oom.conf"
"sudo systemctl status systemd-oomd --full --lines 1000"
Chris Siebenmann's "I've now disabled systemd-oomd on my Fedora desktops"
ArchWiki's "Improving system responsiveness under low-memory conditions"
David Rientjes' "User-space out-of-memory handling"
Kernel OOM-killer: kernel.org's "Out Of Memory Management"
systemd-oomd
"sudo sysctl -a | grep oom"
Add "vm.oom_kill_allocating_task = 0" to /etc/sysctl.conf ?



sudo ps -ex | grep oom

sudo journalctl | grep 'Out of memory'
# To see more, take the timestamp when that occurred, and do:
sudo journalctl | grep 'TIMESTAMP'

sudo systemctl status systemd-oomd --full --lines 1000

# to turn it off, persistently across reboots:
sudo systemctl mask systemd-oomd.service

# Starting with systemd 251, unit status will show "oom-kill" if killed by OOM:
sudo systemctl status UNIT

Memory allocated by all the processes of an application:

# https://opensource.com/article/21/10/memory-stats-linux-smem
smem -c pss -P APPNAME -k -t | tail -n 1
# Doesn't work on Flatpaks.

# "There is a systemd scope for each running app named app-flatpak-$FLATPAKID-$INSTANCEID.scope"
systemctl status --user | grep firefox | grep scope # find scope name
systemctl status --user SCOPENAME   # doesn't show memory use
systemctl show --user -p MemoryAccounting SCOPENAME # says MemoryAccounting is on
systemctl show --user -p MemoryCurrent SCOPENAME    # says "[not set]"

systemd-cgtop   # doesn't show scopes for Flatpak apps

flatpak ps
flatpak enter PROCESSID sh
# Then what ?

There are some ways to control OOM-killing, but it seems there is no simple way to control it per-application (as in "never kill Firefox"). You can control it per-service, or per-process once a process is running and has a PID.
Baeldung's "Linux Memory Overcommitment and the OOM Killer"
Ibrar Ahmed's "How to Adjust Linux Out-Of-Memory Killer Settings"
"The way the systemd OOM killer behaves is to monitor the usage of the entire cgroup, and when the cgroup exceeds its limit, the whole cgroup is killed, not just the most offensive process within the cgroup."


# Display process OOM scores, 0 == don't ever kill:
while read -r pid comm
do
    printf '%d\t%d\t%s\n' "$pid" "$(cat /proc/$pid/oom_score)" "$comm"
done < <(ps -e -o pid= -o comm=) | sort -k2 -n

# Display adjustment and score for one process:
sudo cat /proc/PROCESSID/oom_score_adj   # negative == less likely to be killed
sudo cat /proc/PROCESSID/oom_score
choom -p PROCESSID
choom -p $(pgrep firefox)

# Which processes have adjustments so less likely to kill:
export pids=""
export sep=""
for pid in `grep '-' /proc/*/oom_score_adj | cut -d / -f 3`
do
pids="$pids$sep$pid"
sep=","
done
ps -p $pids
grep '-' /proc/*/oom_score_adj

# Set a process to be harder to kill:
sudo choom -p PROCESSID --adjust -200
sudo choom -p $(pgrep firefox) --adjust -200

# Set application to be harder to kill:
for pid in `pidof APPNAME`
do
sudo echo '-200' >"/proc/$pid/oom_score_adj"
done
# Some processes seem to have hardcoded values,
# will give "permission denied".

To force an out-of-memory condition for testing:
"tail /dev/zero" (looks for newline but never finds one)

There is a system-wide overcommit policy that can be changed.
"cat /proc/sys/vm/overcommit_memory"
See section 5 of Baeldung's "Linux Memory Overcommitment and the OOM Killer"



See if you are using any non-free software

+/-

sudo apt install vrms
# note: installation creates a monthly cron job !
vrms



Using your Linux box to do penetration-testing of other devices




Compiling stuff from source

+/-
LinuxJourney
Chris Hoffman's "How To Compile and Install from Source on Ubuntu"

SK's "An Easy Way To Remove Programs Installed From Source In Linux"
Ubuntu's "CheckInstall"

I don't know how to register a compiled app under apt. But once you've done so, you can create a .deb file from it by using "dpkg-repack". Or:
Debian Packager



"I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."
-- Charles Babbage