Greetings,
On Friday, 2022-03-18 11:50:33 +0100, I myself wrote:
Nikos,
On Thursday, 2022-03-17 19:04:04 +0200, you wrote:
...
http://mmogilvi.users.sourceforge.net/software/oauthbearer.html
...
Really interesting reading. Thanks for the pointer. And also thanks to
the other responders.
Matthew Ogilvie's write-up at the above URL was not only interesting but
also extremely helpful. It basically covers four points:
1. Background information on OAuth 2.
2. A how-to about setting up one's Google mail account for OAuth 2 and a
Python script (requiring a small configuration file), which gets you
your first two OAuth tokens: an "access token" which has to be passed
to Google as a password, but which has only a life time of 60 minut-
es, and a "refresh token" which is valid until you change your Google
password or maybe also until Google decides enough being enough and
starts rejecting your refresh requests with "HTTP Error 400: Bad Re-
quest" messages which you'll find at the bottom of the Python trace-
back from Matthew's script. As long as the refresh token is valid
you will have to use this script on a rather regular basis to get new
valid access tokens.
3. Gentoo-ready patches for "net-mail/fetchmail" up to at least version
6.4.13 which however will only support IMAP and will not work in dae-
mon mode, thus requiring a "cron" job for fetching mail.
Matthew also points out that Gentoo offers a still masked "net-mail/
fetchmail" version 7.0.0_alpha9-r1 which supports OAuth 2, POP3, and
daemon mode.
4. A description how to tweak "mail-mta/postfix" so it supports OAuth 2.
This involves installing a "cyrus-sasl-xoauth2" plugin not available
in the Gentoo repository, a complex configuration setup for a full-
fledged MTA which above all is dynamically changing (at least, if you
want to handle outgoing mail for more than one user), special "sudo-
ers" rules for these mailing users, and "cron" jobs to periodically
send their mail off to their Google accounts.
Following part 2 of this guide I soon had my Google account OAuth-ready.
The Python script from Matthew requires a configuration file the path to
which has to be explicitly specified in every call. My configuration
file resides at "~/.../oauth.cfg" and contains:
client_id=...
client_secret=...
access_token_file=/home/rainer/.../oauth-access-token refresh_token_file=/home/rainer/.../oauth-refresh-token
max_age_sec=1800
The first two lines specify the Google project just created, the next
two lines define the absolute paths to the two token files to be used,
and the last line sets the access token age before which the token will
not be renewed. DO NOT try to add empty or comment lines to this file!
Regarding the adaption of "fetchmail" in part 3 I opted for installing version 7.0.0_alpha9-r1, since I was used to POP3 and daemon mode [1]. Version 7 is still masked, so to be able to emerge it execute (in the
first "echo" command replace "amd64" with YOUR architecture):
# mkdir -p /etc/portage/package.accept_keywords
# echo '>=net-mail/fetchmail-7.0.0_alpha9-r1 ~amd64' \
>> /etc/portage/package.accept_keywords/Oauth
# mkdir -p /etc/portage/package.unmask
# echo '>=net-mail/fetchmail-7.0.0_alpha9-r1' \
>> /etc/portage/package.unmask/OAuth
[1] If you do not like installing an alpha-version, want to use POP3,
but do not insist on daemon mode, I've meanwhile found out that Gen-
too also provides a package "net-mail/mpop" from the same author as
the MTA I decided to install instead of "mail-mta/postfix". Howev-
er, Gentoo-wise this package is almost two years behind due to the
lack of a Gentoo maintainer, so you might want to install it direct-
ly from
https://marlam.de/mpop/
Of course the new "fetchmail" version also introduced new configuration
items. Here is my new "~/.fetchmailrc" file (the "authenticate" direct-
ive as well as the last three lines are new):
set daemon 60
set invisible
set no syslog
poll pop.gmail.com
protocol POP3
service 995
authenticate oauthbearer
username "
rainer.woitok@gmail.com"
dropdelivered
fetchall
no keep
mda "/usr/bin/procmail -pf %F"
passwordfile "/home/rainer/.../oauth-access-token"
sslmode wrapped
sslcertck
Even though I run "fetchmail" in daemon mode, it's not running all the
time. It's started once when I log in, but my backup script which I use
at least once a day will again terminate it before doing anything else,
so my mailbox will never change between my last backup and hibernating
my laptop.
And yes, I almost never shutdown my laptop, but rather suspend or hib-
ernate it. Thus I'll possibly need a fresh access token when I manually
start "fetchmail", but it's not necessary to repeatedly update it via
"cron" while "fetchmail" isn't running at all. The same holds when re-
suming from suspension or hibernation: only when "fetchmail" is running
the age of the access token should be checked and potentially a new one requested.
So I'll have to use Matthew's "oauth.py" script in three ways:
1. Running it the first time or starting over to get both, a new refresh
token and a new access token.
2. Unconditionally checking the age of the access token and optionally
requesting a new one, regardless of whether or not "fetchmail" is ex-
ecuting.
3. Only checking the age of the access token and optionally requesting a
new one, if "fetchmail" is really executing.
This leads to wrapper script "~/.../oauth.sh" which knows the options it
has to pass to script "oauth.py":
#! /bin/bash -u
#
# Script to retrieve new OAuth tokens from Google mail.
#
# There are two types of OAuth tokens, an "access token" which only has
# a life time of 60 minutes and is required for both, fetching mail from
# and sending mail via Google mail, and a "refresh token" which is valid
# until changing one's Google mail password or until Google decides it
# has been used long enough, and is required to get a new "access tok-
# en".
#
# The action performed by this script depends on its first argument:
#
# "init": Get a (new) refresh token together with a new access token.
# This causes an URL to be sent to standard output, which you
# have to point your browser to. At Google's mail login page
# use your mail address and password to log into your mail ac-
# count, agree to accessing your project, copy the authorizat-
# ion code then provided, paste it after the text "Enter ver-
# ification code:" in the terminal window where this script is
# still running, and hit Enter.
#
# "update": Update the access token, except when it is younger than the
# maximum age specified in configuration file "oauth.cfg".
#
# no arg: Only when "fetchmail" is running update the access token as
# if "update" has been specified.
dir=$(dirname $(realpath "$0")) # Symlink-free abs path to script dir. cfg="--config_file=$dir/oauth.cfg" # Configuration file option.
if [[ "${1-}" = init ]]
then "$dir/oauth.py" "$cfg" --obtain_refresh_token_file
elif pgrep -u $USER fetchmail > /dev/null || [[ "${1-}" = update ]]
then "$dir/oauth.py" "$cfg" --auto_refresh
fi
Now keeping the above comment regarding "init" in mind I executed:
$ ~/.../oauth.sh init
to get my first OAuth 2 access tokens. I then created:
1. A new "fetchmail" alias (which will also be run upon login):
$ alias fetchmail='~/.../oauth.sh update ; fetchmail'
2. A "crontab" entry for my own userid (don't forget to add yourself to
group "cron"):
# When "fetchmail" is started, it will request a new "OAuth" access tok-
# en from Google as soon as the current token is older than 30 minutes.
# And as long as "fetchmail" is running, this "cron" job, too, will re-
# new the "OAuth" access token when it is older than 30 minutes. Since
# we start this "cron" job three times an hour, this will result in acc-
# ess tokens (except for the first one, which will be refreshed after 30
# to 50 minutes) being refreshed every 40 minutes. This fits well with
# the maximum life time of 60 minutes for any access token.
11,31,51 * * * * $HOME/.../oauth.sh
3. The hook script "/lib64/elogind/system-sleep/oauth-token.sh" for the
"elogind" service:
#! /bin/bash -u
#
# When resuming from hibernation or suspension update Google's OAuth to-
# ken for user "rainer".
#
# This script will be called by "elogind" with exactly two arguments:
#
# $1: Either "pre" or "post".
# $2: Either "hibernate", "hybrid-sleep", "suspend", or "suspend-then-
# hibernate".
#
# The "sleep" command is required to give the host some time connecting
# the WiFi before running "oauth.sh", and taking this nap in the back-
# ground prevents it from delaying "elogind" while firing up the WiFi:
[[ "$1" = pre ]] || (sleep 10 ; sudo -u rainer ~rainer/.../oauth.sh) &
Now it's time to just start "fetchmail" via its new alias and to catch
up with one's incoming mail ...
However, I did not follow Matthew's suggestions in part 4 regarding the configuration of "mail-mta/postfix" because the configuration effort for
a full-fledged, system wide MTA as well as adding another "ebuild" file
to my local overlay seemed like overkill to me. Up to now I was using "ssmtp" for sending mail, and if I remember correctly, the first "s" in
this name was referring to "simple". Its configuration file just con-
tained eight lines. Looking around I found Gentoo package "mail-mta/ msmtp", which supports OAuth 2 and also a user local configuration file
at "~/.msmtprc". Here are the relevant lines from mine:
# Name the only account defined here "default", so it overrides the def-
# ault account defined in file "/etc/msmtprc":
account default
auth oauthbearer
domain gmail.com
host smtp.gmail.com
passwordeval cat ~/.../oauth-access-token
port 587
protocol smtp
tls on
tls_certcheck on
tls_starttls on
user
rainer.woitok@gmail.com
Now the only thing left to do is telling one's MUA not to pipe outgoing
mail directly to the standard "/usr/sbin/sendmail", but rather to script "~/.../msmtp.sh" instead, so we can refresh the OAuth access token, if necessary, before really sending off the mail. Script "~/.../msmtp.sh" contains:
#! /usr/bin/bash -u
#
# Before sending the mail refresh the access token, if necessary, and
# since the mail to be sent is provided on standard input, route stand-
# ard input around the call to "oauth.sh":
{ cat >&3 ; ~/.../oauth.sh update ; } 3>&1 | msmtp "$@"
Should you MUA not provide a way for specifying the programme to which outgoing mail is piped, you can simply modify the symbolic link "/usr/ sbin/sendmail" (which has just been set by package "mail-mta/msmtp" to
point to "../bin/msmtp"):
# cd /usr/sbin
# rm sendmail
# ln -s ~rainer/.../msmtp.sh sendmail
That's all, happy mailing ... :-)
Sincerely,
Rainer
PS: This mail was sent to you using "mail-mta/msmtp" set-up the way de-
scribed above ... :-)
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)