Show contents of a file without commented out lines and blank lines
A customer asked me how to display the contents of a text file, in
this case a configuration file, without showing any blank line, nor
any lines that started with a "#" character (commented out lines).
We can do this at the Linux command line, using sed,
if you know how to craft the regex:
For further reference: https://www.gnu.org/software/sed/manual/sed.html
this case a configuration file, without showing any blank line, nor
any lines that started with a "#" character (commented out lines).
We can do this at the Linux command line, using sed,
if you know how to craft the regex:
sed -e '/^ *#/d' -e '/^$/d' /dir/file.conf
For further reference: https://www.gnu.org/software/sed/manual/sed.html
There are thousands of resources online: GOOGLE
2018 Nebraska Cornhuskers Football Schedule
DATE OPPONENT RESULT/TIME
============ ============== =================
Sat, Sept 1 Akron Canceled
Sat, Sept 8 Colorado Loss 33-28
Sat, Sept 15 Troy Loss 24-19
Sat, Sept 22 @ #21 Michigan Loss 56-10
Sat, Sept 29 Purdue Loss 42-28
Sat, Oct 6 @ #5 Wisconsin 7:30pm
Sat, Oct 13 @ Northwestern 12:00 PM ET
Sat, Oct 20 Minnesota TBD
Sat, Nov 3 @ #4 Ohio St TBD
Sat, Nov 10 Illinois TBD
Sat, Nov 17 #15 Mich. St. TBD
Fri, Nov 23 @ Iowa 12:00 PM ET FOX
Source
============ ============== =================
Sat, Sept 1 Akron Canceled
Sat, Sept 8 Colorado Loss 33-28
Sat, Sept 15 Troy Loss 24-19
Sat, Sept 22 @ #21 Michigan Loss 56-10
Sat, Sept 29 Purdue Loss 42-28
Sat, Oct 6 @ #5 Wisconsin 7:30pm
Sat, Oct 13 @ Northwestern 12:00 PM ET
Sat, Oct 20 Minnesota TBD
Sat, Nov 3 @ #4 Ohio St TBD
Sat, Nov 10 Illinois TBD
Sat, Nov 17 #15 Mich. St. TBD
Fri, Nov 23 @ Iowa 12:00 PM ET FOX
Source
How to configure Postfix on CentOS 7 to relay email via Gmail
Postfix
is a flexible mail server that is available on most Linux
distributions. Although Postfix is a full feature mail server, it can also be
used as a simple relay host to relay email to another mail server or
smart host for processing. This tutorial will describe how to configure Postfix as a
relay through Gmail.
Simple
Authentication and Security Layer (SASL) is a standard authentication
framework supported by many services including Postfix.
Requirements
- CentOS 7
- A Valid Gmail account or Google App credentials (I recommend creating a credential just for this use)
Install Packages
Ensure Postfix, the SASL authentication framework, and mailx are all installed:
yum -y install postfix cyrus-sasl-plain mailx
Postfix will need to be restarted before the SASL framework will be detected.:
systemctl restart postfix
Postfix should also be set to start on boot:
systemctl enable postfix
Configure Postfix
Open the /etc/postfix/main.cf file in your favorite text editor (vi!) and add the following lines to the end of the file:
myhostname = hostname.example.com #The hostname of your server
relayhost = [smtp.gmail.com]:587
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_tls_CAfile = /etc/ssl/certs/ca-bundle.crt
smtp_sasl_security_options = noanonymous
smtp_sasl_tls_security_options = noanonymous
The myhostname parameter is optional. If the hostname is not specified, Postfix will use the fully-qualified domain name of the Linux server.
Save the main.cf file and close the editor.
Configure Postfix SASL Credentials
The Gmail credentials must now be added for authentication. Create a /etc/postfix/sasl_passwd file and add the following line:
[smtp.gmail.com]:587 username:password
The username and password values must be replaced with valid Gmail credentials. The sasl_passwd file
can now be saved and closed.
A Postfix lookup table must now be generated from the sasl_passwd text file by running the following command:
postmap /etc/postfix/sasl_passwd
Access to the sasl_passwd files should be restricted:
chown root:postfix /etc/postfix/sasl_passwd*
chmod 640 /etc/postfix/sasl_passwd*
And finally, reload the Postfix configuration:
systemctl reload postfix
Test the Relay
Use the mail command to test the relay:
echo "This is a test." | mail -s "test message" user@example.net
Instead of user@example.net, use a valid email address. Check your email!
Troubleshoot Delivery Issues
Postfix will log to /var/log/maillog, and this file can
be reviewed if the test message is not successfully delivered. Open a
separate shell window to watch the maillog using this command:
tail -f /var/log/maillog
Now re-run the email test in the original terminal window. Watch the log entries.
If there are not enough details in the log to determine the problem, then the debug level can be increased by adding the following lines to the /etc/postfix/main.cf file:
debug_peer_list=smtp.gmail.com
debug_peer_level=3
The Postfix configuration must be reloaded after updating the main.cf file:
systemctl reload postfix
Do not leave those settings in the configuration file long term, or your log files will get excessively large, which can have a negative impact on server performance.
Quick & Dirty: How do I get Linux?
Someone recently asked me how they can get Linux, install it on something, and start learning how to use it. It's actually not that difficult to do these days. Here is the high-level view of what is going on:
Download a free CentOS Linux installation ISO file from here:
https://mirror.umd.edu/centos/
Save the file to your Windows Desktop or laptop.
Download Oracle VirtualBox from here: https://www.virtualbox.org/wiki/Downloads
Install VirtualBox on your Windows PC or laptop, following the instructions from their website. There's also a Mac version if you play like that. Once VirtualBox is installed, go back to your download directory and double-click on the Extension Pack to install it. You now have a free virtualization platform on your Windows machine! In the VirtualBox application, you can create a new Virtual Machine, which will run on your computer, and look/act like a real system. You can then install Linux using the ISO image file you downloaded earlier.
Instructions (with pictures) for installing CentOS 7 are available all over the internet. However, I think the person who blogs on this particular site, has done a very good write-up (with pictures):
https://www.server-world.info/
The site also has many tutorials for installing and configuring many different things on Linux.
Go to Google, type in "Linux for beginners" and hit enter. There are tons of free online resources to learn more about Linux!
Download a free CentOS Linux installation ISO file from here:
https://mirror.umd.edu/centos/
Save the file to your Windows Desktop or laptop.
Download Oracle VirtualBox from here: https://www.virtualbox.org/wiki/Downloads
Install VirtualBox on your Windows PC or laptop, following the instructions from their website. There's also a Mac version if you play like that. Once VirtualBox is installed, go back to your download directory and double-click on the Extension Pack to install it. You now have a free virtualization platform on your Windows machine! In the VirtualBox application, you can create a new Virtual Machine, which will run on your computer, and look/act like a real system. You can then install Linux using the ISO image file you downloaded earlier.
Instructions (with pictures) for installing CentOS 7 are available all over the internet. However, I think the person who blogs on this particular site, has done a very good write-up (with pictures):
https://www.server-world.info/
The site also has many tutorials for installing and configuring many different things on Linux.
Go to Google, type in "Linux for beginners" and hit enter. There are tons of free online resources to learn more about Linux!
Check SUID/SGID program Audit rules
Here's an interesting BASH script that checks that all SUID/SGID files have a corresponding audit rule. Useful if you work in an environment where you are required to periodically perform this check, and like all good (lazy) sysadmins, wish you had a script to automate this task...
exit 0
#!/bin/bash
for i in `lsblk --output MOUNTPOINT | grep /`
do
for i in `lsblk --output MOUNTPOINT | grep /`
do
for j in `find ${i} -xdev -type f \( -perm -4000 -o -perm -2000 \)`
do
if [ "$(grep -c ${j} /etc/audit/audit.rules)" -ge 1 ]; then
# If there is a rule...do nothing?
do
if [ "$(grep -c ${j} /etc/audit/audit.rules)" -ge 1 ]; then
# If there is a rule...do nothing?
echo "" > /dev/null
else
echo "suid/sgid program ${j} - NO audit rule!"
fi
done
done
else
echo "suid/sgid program ${j} - NO audit rule!"
fi
done
done
Display the contents of a config file without commented out or blank lines
"Hi Pete! Is there a way that I can easily display the contents of a Linux config file, without showing all the commented out lines, and all the blank lines?"
This is easy to do with sed. Keep this one-line-command in your notes:
This is easy to do with sed. Keep this one-line-command in your notes:
sed -e '/^ *#/d' -e '/^$/d' /etc/httpd/conf/httpd.conf
We're telling the sed command to display a file to STDOUT after performing
two edits (-e) on the output stream. The first one deletes all lines that
start with a pound sign (#), and the second edit deletes all blank lines.
These deletes are not changes to the file itself! All that is changed is
the data sent to STDOUT.
Installed RPM inventory
"I need a list of all installed RPMs on my CentOS server. But I want it as a CSV file (comma separated values). How can I do this in BASH?"
Here's the command (all on one line) that will produce the output that you want:
rpm -qa --qf "\"$(uname -n)\",\"%{name}\",\"%{version}-%{release}\",\"%{vendor}\",\"%{license}\",\"%{summary}\"\n" > /tmp/software-list.csv
Now you can scp the file to your laptop, then open the file using MS-Excel (or your favorite spreadsheet program). It's been my experience that the vast majority of corporate laptop images are a version of Microsoft Windows. The only company I have ever worked for where this was not the case was Red Hat Inc.
Here's the command (all on one line) that will produce the output that you want:
rpm -qa --qf "\"$(uname -n)\",\"%{name}\",\"%{version}-%{release}\",\"%{vendor}\",\"%{license}\",\"%{summary}\"\n" > /tmp/software-list.csv
Now you can scp the file to your laptop, then open the file using MS-Excel (or your favorite spreadsheet program). It's been my experience that the vast majority of corporate laptop images are a version of Microsoft Windows. The only company I have ever worked for where this was not the case was Red Hat Inc.
A script to update a CentOS7 YUM Repository
Here's a BASH script to use to update a CentOS7 YUM Repository:
#!/bin/bash
# A script to update a CentOS7 Yum Repository
REPOS=(base updates extras)
for REPO in ${REPOS[@]}
do
echo "Updating repository: ${REPO} "
/usr/bin/reposync --repoid=${REPO} --download_path=/repo -g -n -l -m
if [ -e /repo/${REPO}/comps.xml ]; then
/usr/bin/createrepo -v -g /repo/${REPO}/comps.xml /repo/${REPO}/
else
/usr/bin/createrepo -v /repo/${REPO}/
fi
done
chown -R apache:apache /repo/*
restorecon -Rv /repo/*
exit 0
Generating a public/private keypair the right way
From the command line:
# ssh-keygen -t rsa -b 4096 -o -a 100
-a rounds
When saving a new-format private key (i.e. an
ed25519 key or any SSH protocol 2 key when the
-o flag is set), this option specifies the number
of KDF (key derivation function) rounds used.
Higher numbers result in slower passphrase
verification and increased resistance to brute-force
password cracking (should the keys be stolen).
-b bits
Specifies the number of bits in the key to create.
For RSA keys, the minimum size is 1024 bits and the
default is 2048 bits. Generally, 2048 bits is considered
sufficient. DSA keys must be exactly 1024 bits as
specified by FIPS 186-2.
-o Causes ssh-keygen to save private keys
using the new OpenSSH format rather than the
more compatible PEM format. The new format
has increased resistance to brute-force
password cracking but is not supported by
versions of OpenSSH prior to 6.5. Ed25519
keys always use the new private key format.
-t dsa | ecdsa | ed25519 | rsa | rsa1
Specifies the type of key to create. The possible
values are “rsa1” for protocol version 1 and “dsa”,
“ecdsa”, “ed25519”, or “rsa” for protocol version 2.
A customer recently asked me, "Is there a way to look at a configuration file, without all the comments and excess blank lines?"
Yes, there is! You can use the sed utility to display the contents of a file, omitting any blank lines, and omitting any lines that start with the "#" character (which denotes that the line is a comment), like this:
sed -e '/^ *#/d' -e '/^$/d' /etc/httpd/conf/httpd.conf
In this example, I am using the sed command to display the content of the httpd.conf file. The -e option allows me to apply an edit to the stream of output. The first edit ('/^ *#/d') instructs sed to search the output for any lines starting with "#" and delete them. The second edit instructs sed to look for blank lines and delete them. It helps to understand basic Regular Expressions, which is an entire other discussion.
Given a list of hostnames (one per line) in a file named hosts-list, ssh into each server in turn, sudo to root, and execute a command with sudo permissions, You will need to type in the password twice, unless you use an utility like Keepass, that allows you to copy/paste your password. This method makes it very quick and convenient to go thru a long list of servers, to perform a simple admin task.
for name in $(cat hosts-list); do (ssh -t -o StrictHostKeyChecking=no my.username@${name} 'sudo su - -c "uname -a ; yum clean all"' 2> /dev/null) ; done
A BASH script to monitor disk usage in Linux
Someone asked me if disk space usage can be monitored via a bash script. Yes, it can. The following script can be copied to the host to be monitored, then executed periodically (once an hour?) via a crontab. Keep in mind there are bigger better ways out there to do this, but this isn't bad for a small simple solution. It's also a good exercise for you if you are learning Linux!
#!/bin/bash
# A script to keep an eye on disk use. Mutt must be installed
# A script to keep an eye on disk use. Mutt must be installed
# and the host must be able to send SMTP mail.
# Alert Recipient
ADMIN="linux-admin@example.com"
# Alert Recipient
ADMIN="linux-admin@example.com"
# Alert Threshhold
ALERT=85
df -h -P | grep -vE '^Filesystem|tmpfs|cdrom|iso|nfs|140.139' | awk '{ print $5 " " $1 }' | while read OUTPUT;
do
USAGE=$(echo $OUTPUT | awk '{ print $1 }' | cut -d'%' -f1 )
PARTITION=$(echo $OUTPUT | awk '{ print $2 }')
if [ $USAGE -ge $ALERT ] ; then
echo -e "WARNING: Filesystem on \"$PARTITION\" is ${USAGE}% full.\n Threshold is 85% \n HOST: $HOSTNAME\nDATE: $(date)\nThis message generated by /admin-scripts/SpaceWatch on $HOSTNAME" | \
mutt -s "Alert: Disk space warning on $HOSTNAME" ${ADMIN}
fi
done
exit 0
df -h -P | grep -vE '^Filesystem|tmpfs|cdrom|iso|nfs|140.139' | awk '{ print $5 " " $1 }' | while read OUTPUT;
do
USAGE=$(echo $OUTPUT | awk '{ print $1 }' | cut -d'%' -f1 )
PARTITION=$(echo $OUTPUT | awk '{ print $2 }')
if [ $USAGE -ge $ALERT ] ; then
echo -e "WARNING: Filesystem on \"$PARTITION\" is ${USAGE}% full.\n Threshold is 85% \n HOST: $HOSTNAME\nDATE: $(date)\nThis message generated by /admin-scripts/SpaceWatch on $HOSTNAME" | \
mutt -s "Alert: Disk space warning on $HOSTNAME" ${ADMIN}
fi
done
exit 0
Subscribe to:
Posts (Atom)