Index: branches/fc15-dev/server/doc/install-howto.sh =================================================================== --- branches/fc15-dev/server/doc/install-howto.sh (revision 2019) +++ branches/fc15-dev/server/doc/install-howto.sh (revision 2046) @@ -7,22 +7,6 @@ # [WIZARD] Semi-production server that will only have # daemon.scripts-security-upd bits, among other -# restricted permissions bits, among other -# restricted permissions bits, among other -# restricted permissions bits, among other # restricted permissions # [TESTSERVER] Completely untrusted server - -set -e -x - -# Some commands should be run as the scripts-build user, not root. - -alias asbuild="sudo -u scripts-build" - -# Old versions of this install document advised setting -# NSS_NONLOCAL_IGNORE=1 anytime you're setting up anything, e.g. using -# yum, warning that useradd will query LDAP in a stupid way that makes -# it hang forever. As of Fedora 13, this does not seem to be a problem, -# so it's been removed from the instructions. If an install is hanging, -# though, try adding NSS_NONLOCAL_IGNORE. # This is actually just "pick an active scripts server". It can't be @@ -41,68 +25,7 @@ server=YOUR-SERVER-NAME-HERE -# Start with a Scripts kickstarted install of Fedora (install-fedora) - -# Take updates, reboot if there's a kernel update. - yum update -y - -# Get rid of network manager - yum remove NetworkManager - -# Copy over root's dotfiles from one of the other machines. -# Perhaps a useful change is to remove the default aliases - cd /root - ls -l .bashrc - ls -l .screenrc - ls -l .ssh - ls -l .vimrc - ls -l .k5login - # [PRODUCTION] This rc file has sensitive data on it and should only - # be pushed onto production servers. - ls -l .ldapvirc - # Trying to scp from server to server won't work, as scp - # will attempt to negotiate a server-to-server connection. - # Instead, scp to your trusted machine as a temporary file, - # and then push to the other server -scp -r root@$source_server:~/{.bashrc,.screenrc,.ssh,.vimrc,.k5login} . -scp -r {.bashrc,.screenrc,.ssh,.vimrc,.k5login} root@$server:~ -# [PRODUCTION] -scp root@$source_server:~/.ldapvirc . -scp .ldapvirc root@$server:~ - -# Install the initial set of credentials (to get Kerberized logins once -# krb5 is installed). Otherwise, SCP'ing things in will be annoying. -# Note that if you don't feel like forcefully ignoring "public key -# changed" errors, all you need to transfer is /etc/ssh/ssh_host_rsa_key -# initially. -# o Install the machine keytab. - ls -l /etc/krb5.keytab -# Use ktutil to combine the host/scripts.mit.edu and -# host/scripts-vhosts.mit.edu keys with host/this-server.mit.edu in -# the keytab. Do not use 'k5srvutil change' on the combined keytab -# or you'll break the other servers. (real servers only). Be -# careful about writing out the keytab: if you write it to an -# existing file the keys will just get appended. The correct -# credential list should look like: -# ktutil: l -# slot KVNO Principal -# ---- ---- --------------------------------------------------------------------- -# 1 5 host/old-faithful.mit.edu@ATHENA.MIT.EDU -# 2 3 host/scripts-vhosts.mit.edu@ATHENA.MIT.EDU -# 3 2 host/scripts.mit.edu@ATHENA.MIT.EDU -# o [PRODUCTION] Replace the ssh host keys with the ones common to all -# scripts servers (real servers only) - ls -l /etc/ssh/*key* -# You can do that with: -scp root@$source_server:/etc/ssh/*key* . -scp *key* root@$server:/etc/ssh/ - # Actually, this appears to be unnecessary - service sshd reload - -# Check out the scripts /etc configuration - # backslash to make us not use the alias - cd /root - \cp -a etc / - chmod 0440 /etc/sudoers - +# ----------------------------->8-------------------------------------- +# FIRST TIME INSTRUCTIONS +# # [PRODUCTION] If this is the first time you've installed this hostname, # you will need to update a bunch of files to add support for it. These @@ -124,24 +47,53 @@ # o Set up Nagios monitoring on sipb-noc for the host # o Set up the host as in the pool on r-b/r-b /etc/heartbeat/ldirectord.cf - XXX TODO COMMANDS - -# NOTE: You will have just lost DNS resolution and the ability -# to do password SSH in. If you managed to botch this step without -# having named setup, you can do a quick fix by frobbing /etc/resolv.conf -# with a non 127.0.0.1 address for the DNS server. Be sure to revert it once -# you have named. - -# NOTE: You can get password SSH back by editing /etc/ssh/sshd_config (allow -# password auth) and /etc/pam.d/sshd (comment out the first three auth -# lines). However, you should have the Kerberos credentials in place -# so as soon as you install the full set of Scripts packages, you'll get -# Kerberized logins. - -# Make sure network is working. If this is a new server name, you'll -# need to add it to /etc/hosts and -# /etc/sysconfig/network-scripts/route-eth1. Kickstart should have +# o Update locker/etc/known_hosts +# +# You will also need to prepare the keytabs for credit-card. In particular, +# use ktutil to combine the host/scripts.mit.edu and +# host/scripts-vhosts.mit.edu keys with host/this-server.mit.edu in +# the keytab. Do not use 'k5srvutil change' on the combined keytab +# or you'll break the other servers. (real servers only). Be +# careful about writing out the keytab: if you write it to an +# existing file the keys will just get appended. The correct +# credential list should look like: +# ktutil: l +# slot KVNO Principal +# ---- ---- --------------------------------------------------------------------- +# 1 5 host/old-faithful.mit.edu@ATHENA.MIT.EDU +# 2 3 host/scripts-vhosts.mit.edu@ATHENA.MIT.EDU +# 3 2 host/scripts.mit.edu@ATHENA.MIT.EDU +# +# The LDAP keytab should be by itself, so be sure to delete it and +# put it in its own file. +# ----------------------------->8-------------------------------------- + +# Start with a Scripts kickstarted install of Fedora (install-fedora) + +# Take updates, reboot if there's a kernel update. + yum update -y + +# Get rid of network manager + yum remove NetworkManager + +# This is superseded by credit-card, but only for [PRODUCTION] +# Don't use credit-card on [WIZARD]: it will put in the wrong creds! +# +# # All types of servers will have an /etc/daemon.keytab file, however, +# # different types of server will have different credentials in this +# # keytab. +# # [PRODUCTION] daemon.scripts +# # [WIZARD] daemon.scripts-security-upd +# # [TESTSERVER] daemon.scripts-test + +# Check out the scripts /etc configuration + cd /root + \cp -a etc / + chmod 0440 /etc/sudoers + +# Make sure network is working. Kickstart should have # configured eth0 and eth1 correctly; use service network restart -# to add the new routes in route-eth1. - service network restart +# to add the new routes from etc in route-eth1. + systemctl restart network.service + # Check everything worked: route ifconfig @@ -160,23 +112,5 @@ rpm -e --nodeps rsyslog yum install -y syslog-ng - chkconfig syslog-ng on - -# [PRODUCTION/WIZARD] Fix the openafs /usr/vice/etc <-> /etc/openafs -# mapping. - echo "/afs:/usr/vice/cache:10000000" > /usr/vice/etc/cacheinfo - echo "athena.mit.edu" > /usr/vice/etc/ThisCell - -# [TESTSERVER] If you're installing a test server, this needs to be -# much smaller; the max filesize on XVM is 10GB. Pick something like -# 500000. Also, some of the AFS parameters are kind of retarded (and if -# you're low on disk space, will actually exhaust our inodes). Edit -# these parameters in /etc/sysconfig/openafs - echo "/afs:/usr/vice/cache:500000" > /usr/vice/etc/cacheinfo - XXX TODO COMMANDS - -# Test that zephyr is working - chkconfig zhm on - service zhm start - echo 'Test!' | zwrite -d -c scripts -i test + systemctl enable syslog-ng.service # Install the full list of RPMs that users expect to be on the @@ -216,7 +150,11 @@ rpm -i ghc-cgi*1.8.1*.rpm -# Check out the scripts /usr/vice/etc configuration - cd /root/vice - \cp -a etc /usr/vice +# Note: Since ultimately we'd like to move away from using per-language +# package manager and all of these be RPMs, it is of questionable +# importance how much /good/ automation for these is necessary. + +# Warning: For a new release, we're supposed to check if Fedora has +# packaged up the RPM. Unfortunately we don't really have good incants +# for this. # Install the full list of perl modules that users expect to be on the @@ -246,12 +184,17 @@ # want to be able to write to ~/.python-eggs. (Also makes sourcediving # easier.) -cat /usr/lib/python2.6/site-packages/easy-install.pth | grep "^./" | cut -c3- | cut -f1 -d- > egg.txt +# 'easy_install AuthKit jsonlib2 pygit' +cat /usr/lib/python2.7/site-packages/easy-install.pth | grep "^./" | cut -c3- | cut -f1 -d- > egg.txt cat egg.txt | xargs easy_install -Z + # - Look at `gem list` for Ruby gems. # Again, use 'yum search' and prefer RPMs, but failing that, 'gem install'. # ezyang: rspec-rails depends on rspec, and will override the Yum # package, so... don't use that RPM yet +# XXX This doesn't do the right thing for old version gems gem list --no-version > gem.txt gem install $(gem list --no-version | grep -Fxvf - gem.txt) + # Also, we need to install the old rails version + # - Look at `pear list` for Pear fruits (or whatever they're called). # Yet again, 'yum search' for RPMs before resorting to 'pear install'. Note @@ -262,4 +205,5 @@ pear channel-update pear.php.net pear install $(pear list | tail -n +4 | cut -f 1 -d " " | grep -Fxvf - pear.txt) + # - Look at `pecl list` for PECL things. 'yum search', and if you must, # 'pecl install' needed items. If it doesn't work, try 'pear install @@ -268,40 +212,109 @@ pecl install --nodeps $(pecl list | tail -n +4 | cut -f 1 -d " " | grep -Fxvf - pecl.txt) -# [PRODUCTION] Install the credentials. There are a lot of things to -# remember here. Be sure to make sure the permissions match up (ls -l -# on an existing server!). -scp root@$source_server:{/etc/{sql-mit-edu.cfg.php,pki/tls/private/scripts.key,signup-ldap-pw,whoisd-password},/home/logview/.k5login} . -scp signup-ldap-pw whoisd-password sql-mit-edu.cfg.php root@$server:/etc -scp scripts.key root@$server:/etc/pki/tls/private -scp .k5login root@$server:/home/logview -# o The SSL cert private key (real servers only) - ls -l /etc/pki/tls/private/scripts.key -# o The LDAP password for the signup process (real servers only) - ls -l /etc/signup-ldap-pw -# o The whoisd password (real servers only) - ls -l /etc/whoisd-password -# o Make sure logview's .k5login is correct (real servers only) - cat /home/logview/.k5login - -# All types of servers will have an /etc/daemon.keytab file, however, -# different types of server will have different credentials in this -# keytab. -# [PRODUCTION] daemon.scripts -# [WIZARD] daemon.scripts-security-upd -# [TESTSERVER] daemon.scripts-test -k5srvutil list -f daemon.keytab -scp daemon.keytab root@$server:/etc - chown afsagent:afsagent /etc/daemon.keytab -# o The daemon.scripts keytab (will be daemon.scripts-test for test) - ls -l /etc/daemon.keytab - -# Spin up OpenAFS. This will fail if there's been a new kernel since -# when you last tried. In that case, you can hold on till later to -# start OpenAFS. This will take a little bit of time; - service openafs-client start -# Then, check that fs sysname is correct. You should see, among others, -# 'amd64_fedoraX_scripts' (vary X) and 'scripts'. If it's not, you -# probably did a distro upgrade and should update /etc/sysconfig/openafs. +# ----------------------------->8-------------------------------------- +# INFINITE CONFIGURATION + +# Run credit-card to clone in credentials and make things runabble +python host.py push $server + +# [PRODUCTION/WIZARD] Fix the openafs /usr/vice/etc <-> /etc/openafs +# mapping. + echo "/afs:/usr/vice/cache:10000000" > /usr/vice/etc/cacheinfo + echo "athena.mit.edu" > /usr/vice/etc/ThisCell +# [TESTSERVER] If you're installing a test server, this needs to be +# much smaller; the max filesize on XVM is 10GB. Pick something like +# 500000. Also, some of the AFS parameters are kind of retarded (and if +# you're low on disk space, will actually exhaust our inodes). Edit +# these parameters in /etc/sysconfig/openafs + echo "/afs:/usr/vice/cache:500000" > /usr/vice/etc/cacheinfo + XXX TODO COMMANDS + +# Test that zephyr is working + systemctl enable zhm.service + systemctl start zhm.service + echo 'Test!' | zwrite -d -c scripts -i test + +# Check out the scripts /usr/vice/etc configuration + cd /root/vice + \cp -a etc /usr/vice + +# [PRODUCTION] Set up replication (see ./install-ldap). +# You'll need the LDAP keytab for this server: be sure to chown it +# fedora-ds after you create the fedora-ds user + ls -l /etc/dirsrv/keytab + cat install-ldap + + systemctl enable openafs-client.service + systemctl enable dirsrv.service + systemctl enable nslcd.service + systemctl enable nscd.service + systemctl enable postfix.service + systemctl enable nrpe.service + systemctl enable httpd.service # not for [WIZARD] + + systemctl start openafs-client.service + systemctl start dirsrv.service + systemctl start nslcd.service + systemctl start nscd.service + systemctl start postfix.service + systemctl start nrpe.service + systemctl start httpd.service # not for [WIZARD] + +# Note about OpenAFS: Check that fs sysname is correct. You should see, +# among others, 'amd64_fedoraX_scripts' (vary X) and 'scripts'. If it's +# not, you probably did a distro upgrade and should update +# /etc/sysconfig/openafs (XXX this is wrong: figuring out new +# systemd world order). fs sysname + +# Postfix doesn't actually deliver mail; fix this + cd /etc/postfix + postmap virtual + +# Munin might not be monitoring packages that were installed after it + munin-node-configure --suggest --shell | sh + +# Run fmtutil-sys --all, which does something that makes TeX work. +# (Note: this errors on XeTeX which is ok.) + fmtutil-sys --all + +# Ensure that PHP isn't broken: + mkdir /tmp/sessions + chmod 01777 /tmp/sessions + # XXX: this seems to get deleted if tmp gets cleaned up, so we + # might need something a little better (maybe init script.) + +# Fix etc by making sure none of our config files got overwritten + cd /etc + svn status -q + # Some usual candidates for clobbering include nsswitch.conf and + # sysconfig/openafs + # [WIZARD/TEST] Remember that changes you made should not get + # reverted! + +# ThisCell got clobbered, replace it with athena.mit.edu + echo "athena.mit.edu" > /usr/vice/etc/ThisCell + +# Reboot the machine to restore a consistent state, in case you +# changed anything. (Note: Starting kdump fails (this is ok)) + +# When all is said and done, fix up the Subversion checkouts + cd /etc + svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/ + cd /usr/vice/etc + svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/ + cd /srv/repository + # Some commands should be run as the scripts-build user, not root. + alias asbuild="sudo -u scripts-build" + asbuild svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/ + asbuild svn up # verify scripts.mit.edu works + +# ------------------------------->8------------------------------- +# ADDENDA AND MISCELLANEOUS THINGS + +# [OPTIONAL] Your machine's hostname is baked in at install time; +# in the rare case you need to change it: it appears to be in: +# o /etc/sysconfig/network +# o your lvm thingies; probably don't need to edit # [WIZARD/TESTSERVER] If you are setting up a non-production server, @@ -338,75 +351,4 @@ vim /home/afsagent/renew # replace all mentions of daemon.scripts.mit.edu -# [PRODUCTION] Set up replication (see ./install-ldap). -# You'll need the LDAP keytab for this server: be sure to chown it -# fedora-ds after you create the fedora-ds user - ls -l /etc/dirsrv/keytab - cat install-ldap - -# Make the services dirsrv, nslcd, nscd, postfix, and httpd start at -# boot. Run chkconfig to make sure the set of services to be run is -# correct. - service nslcd start - service nscd start - service postfix start - chkconfig nslcd on - chkconfig nscd on - chkconfig postfix on - -# [PRODUCTION] - chkconfig dirsrv on - -# [PRODUCTION/TESTSERVER] -# (Maybe WIZARD too once we start doing strange things to autoupgrade -# installs behind firewalls.) - service httpd start # will fail if AFS is not running - chkconfig httpd on - -# nrpe is required for nagios alerts - chkconfig nrpe on - -# [PRODUCTION] Check sql user credentials (needs to be done after LDAP -# is setup) - chown sql /etc/sql-mit-edu.cfg.php - -# Postfix doesn't actually deliver mail; fix this - cd /etc/postfix - postmap virtual - -# Munin might not be monitoring packages that were installed after it - munin-node-configure --suggest --shell | sh - -# Run fmtutil-sys --all, which does something that makes TeX work. -# (Note: this errors on XeTeX which is ok.) - fmtutil-sys --all - -# Ensure that PHP isn't broken: - mkdir /tmp/sessions - chmod 01777 /tmp/sessions - # XXX: this seems to get deleted if tmp gets cleaned up, so we - # might need something a little better (maybe init script.) - -# Ensure fcgid isn't broken (should be 755) - ls -ld /var/run/mod_fcgid - -# Fix etc by making sure none of our config files got overwritten - cd /etc - svn status -q - # Some usual candidates for clobbering include nsswitch.conf and - # sysconfig/openafs - # [WIZARD/TEST] Remember that changes you made should not get - # reverted! - -# ThisCell got clobbered, replace it with athena.mit.edu - echo "athena.mit.edu" > /usr/vice/etc/ThisCell - -# Reboot the machine to restore a consistent state, in case you -# changed anything. (Note: Starting kdump fails (this is ok)) - -# [OPTIONAL] Your machine's hostname is baked in at install time; -# in the rare case you need to change it: it appears to be in: -# o /etc/sysconfig/network -# o your lvm thingies; probably don't need to edit - # [TESTERVER] # - You need a self-signed SSL cert or Apache will refuse to start @@ -421,12 +363,2 @@ # be an accepted vhost name # - Look at the old test server and see what config changes are floating around - -# XXX: our SVN checkout should be updated to use scripts.mit.edu -# (repository and etc) once serving actually works. - cd /etc - svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/ - cd /usr/vice/etc - svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/ - cd /srv/repository - asbuild svn switch --relocate svn://$source_server/ svn://scripts.mit.edu/ - asbuild svn up # verify scripts.mit.edu works Index: branches/fc15-dev/server/doc/upgrade-tips =================================================================== --- branches/fc15-dev/server/doc/upgrade-tips (revision 2019) +++ branches/fc15-dev/server/doc/upgrade-tips (revision 2046) @@ -201,2 +201,14 @@ extensions other than the few we’ve whitelisted should be disabled by emptying their .ini files in /etc/php.d. + +8. Sending announcements +------------------------ + +Once development work has finished, we need to allow users to test +their websites on the new servers. + + SIPB Internal Testing: Send an email to scripts-team@mit.edu + and -c sipb notifying them of testing procedure and known + issues. + + General Testing: