Using the date command in cygwin or Linux

Here was my problem:

I had a Microsoft SQL backup file named “filenamehere_backup_2012_04_13_00646736_7384937.bak” and instead of adjusting the backup file name I thought, can I work around it with a script?  In Windows there’s a “time” command.  If you type in “time /?” you’ll see it’s not exactly the most robust command for time data.  You can also use a command called “date” or “date /?” for the options there.  Again, not the most robust set of tools for playing with date data on the command line.  I’m sure there’s other M$ tools like VB scripting, etc but I’m not a VB guy so…moving on.

So how could I get the date option that’s variable and work with it better?  Cygwin!  Cygwin is  “a collection of tools which provide a Linux look and feel environment for Windows” (Ref the Cygwin Site).  So Cygwin allows you to run Linux binary files (exe’s) in Windows like a tool we’ll use to get our date that we need above “2012_04_13″.

Once you install Cywin (see other sites for that ):

::  Put this script in a file called “daily.sh” or whatever.sh
::  Add it to your cron jobs “crontab -l” to list them, “crontab -e” to edit it and add your path + your .sh script to it.
::  Also note that you need to install “cron” in Cygwin, you have to look for it.  To use Nano as your default editor
::  You need to type into the cygwin command prompt”export EDITOR=nano”, so when you type “crontab -e” your default
::  editor will be nano which is far easier than vi (IMHO).
::  See crontab for more information should you need information on how to schedule scripts in crontab.  I have
::  some stuff about it here -> Link, where I talk about Zimbra backups if you need an idea.

daily.sh file contents:

#!/bin/sh

# Setting the current date variable
curtime=$(date +”%Y”_”%m”_”%d”)
# This yields: 2012_04_12 #For example, year, month, day
prevtime=$(date –date ‘-1 day’ +”%y”_”%m”_”%d”)
# Yields: Current date “2012_04_12″ minus 1 day so result = “2012_04_11”
# Also note that %y yields just “12″ not “2012″

# Troubleshooting here… I like to “echo” the results before putting them into production so I know what I’ m going
# to get.  Simply delete the “#” if you want to run the line, the “#” is a comment, anything after it won’t run.
# echo Current date is
# echo $curtime
# echo File to move today is
# echo “filenamehere_backup_”$curtime”_*.bak”
# echo 1 day ago date is
# echo $prevtime
# echo FIle to delete today is
# echo “filenamehere_backup_”$prevtime”_*.bak”

# Production commands
cp /cygdrive/y/filenamehere_backup_”$curtime”_*.bak
/cygdrive/d/sql2daily
rm -f /cygdrive/d/sql2daily/filenamehere_backup_”$prevtime”_*.bak

Now what did we do in this script and what’s the point of using the date command?  I copied (cp) from the mapped “y:” drive which is a UNC path mapped drive to local disk d: where the tape backup looks to in the backups schedule (I added that folder specifically for this task).

“curtime” gives us the current date, like I’ve commented in my script above:  curtime=$(date +”%Y”_”%m”_”%d”)
# This yields: 2012_04_12 #For example, year, month, day.  If you type in the cygwin terminal (or a Linux terminal) “date –help” you’ll see a plethora of options, far more than in windows.

Next we have “prevtime=$(date –date ‘-1 day’ +”%y”_”%m”_”%d”)”, this takes the date and subtracts 1 day so the 12th becomes the 11th.  Note you can call it anything you want, I called them “curtime” and “prevtime” just because, you could call it peanut and butter if you wanted, they’re arbitrary.  Also note that if you want “10 days” simply change the ‘-1 day’ to ‘-10 days’ and it’ll kick it back 10 days.

* By the way, I notice the way my blog throws in the text that some stuff is a bit weird when you’re looking at it for example the “date – - date” on the published side looks like one dash “-” but there are actually two dashes “date dash dash date”.  So when you do a “date –date”…you need two dashes, there are two dashes there!

When I copy the current days backup into the folder by tomorrow it’s old and the script will copy the new days backup into the directory then “rm” or remove yesterdays copy so all I have in the folder is one days backup!  Lets say you wanted to keep 7 days worth of backups right?  Then you’d change the ‘-1 day’ to ‘-7 day’ or 30 days, etc, etc.  For me it’s the solution I was looking for IF your file has the date in the file name.  If it didn’t well then…you’ll need to do other stuff related to the “ls -la” command I’m sure and store your output into a file or a variable…I’d have to think about how you’d accomplish that BUT the “date” command worked wonderfully for what I needed it for!

* A bit of an edit to my post – I and others online have found it difficult to get cygwin and cron properly working on Windows, it seems to be hit or miss.  What I found and what’s working for me on Windows is bypassing cron by just calling your shell script from a batch file.  Here’s the code that worked for me:

c:
cd\
cygwin\bin\sh.exe -l /home/Administrator/daily.sh

The reason I have the “c:” in there is because I call the script from a D: drive and since cygwin is installed on c:\cygwin, if you don’t change drives in your bat file it doesn’t work

 

Update:

If you’re on a Windows machine and you’re running a batch file that calls rsync or some other process via sh.exe and you encounter the error in your Schedules Task: 0×17

Check your folder permissions – it won’t run because you’re getting access denied on creating directories and files.  Run the script manually by double clicking on the batch file – you’ll see the error, you might have to put a “pause” statement in your batch file too so it won’t terminate.  I ran into this issue because the user account I was running the script under isn’t “administrator” – running your stuff as administrator probably isn’t a “best practice”…even though in practical terms there is no such thing (in my opinion).  I guess “best practice” is a discussion for another time

Manually updating Microsoft Security Essentials from a UNC path

I haven’t seen anyone with this solution yet and I did get a request to post the how to.  A while back I posted on how to push out the Microsoft Security Essentials (MSE) software to any number of machines with a system startup script.  For IT shops on a budget this is a good solution.  Couple this with not having your users running as local administrators or power users, some decent group policies, possibly some firewall packet filtering or AV scanning of that data with a web filter and you’ll be pretty secure!

Depending on your OS (32 bit or 64) you’ll need to adjust these commands to include the proper paths.  For my examples they’re all done on a 32 bit machine so adjust accordingly.

The location of MSE is:

C:\Program Files\Microsoft Security Client\Antimalware

Doing a directory listing we find and notice the bolded files:

C:\Program Files\Microsoft Security Client\Antimalware>dir /B
Drivers
EN-US
IpsConsumer.dll
MpAsDesc.dll
MpClient.dll
MpCmdRun.exe
MpCommu.dll
mpevmsg.dll
MpOAv.dll
MpRTP.dll
MpSvc.dll
MpUtil.dll
MsMpCom.dll
MsMpEng.exe
MsMpLics.dll
NisIpsPlugin.dll
NisLog.dll
NisNetIP.dll
NisPerformanceProvider.dll
NisRes.dll
NisSrv.exe
NisWFP.dll

I’ve found that in the past just messing around with exe files by typing in (for example), mpcmdrun /? – you might find interesting things

Donig so with the command line version of MSE you see you can do all kinds of interesting commands, if you use a program to execute commands on remote machines you can even scan machines remotely!

The file usage is MpCmdRun.exe [command] [-options]

So if we want to do a signature update according to the command options we’d do this:

mpcmdrun -SignatureUpdate -UNC%storepath%

* Of course you must declare your store path as a UNC file path \\servername\share for example

Here’s the batch file I have:

@echo off
:: Setting system variables
set storepath=”\\server\share\distribute\Microsoft_Security_Essentials\updates”

if exist “%storepath%\mpavdlta.vdm” (goto execute)
echo Update package does not exist…
goto end
echo.

:execute
:: Executing package
echo Executing Microsoft Security Essentials Update…
echo.
:: %storepath%\%package%
mpcmdrun -SignatureUpdate -UNC%storepath%

:end

What this does is sets the store path variable, the store path where we will be storing our update.  When you download an update it seems that MSE looks for a file named “mpavdlta.vdm”.   You just have to point it at your UNC share and it automatically looks for that file, I found this out by trial and error!

* Copy the above script into a bat file and make that a user log on script so the user executes the update.  It’s doing the same thing as a user would who opens MSE and manually clicks the “update” button but if it’s a user logon script (preferably via group policy) then it’s a no brainer for the user and you’re telling MSE where to get the update from (NOT your WAN ).

So now; how do we get our “update server” to always have a fresh ready to go update?

You need to use wget for Windows and make a scheduled task on your server that uses wget to download the latest update to the update folder.  You’ll also need 7zip to unzip the package.

Here’s what I have for a script:

:: Here’s the KB on manually downloading the update:  http://support.microsoft.com/kb/971606

@echo off

echo Setting system variables…
echo.
set storepath=”\\server\share\distribute\Microsoft_Security_Essentials\updates”
set patch=mpam-fe.exe
set updateurl=http://download.microsoft.com/download/DefinitionUpdates

:: Mapping drive
echo Mapping x: drive temporarily…
echo.
net use x: %storepath%
echo.

:: Removing the old update file
echo Removing the old udpated file…
echo.
del /q %storepath%\%patch%
echo.

:: Downloading the update
echo Downloading the MSE update…
echo.
%storepath%\wget -P x: “%updateurl%/%patch%”
echo.

:: Unmapping drive
echo Unmapping x: drive…
echo.
net use x: /delete /yes
echo.

:: Unzipping package
echo Unzipping the update…
%storepath%\7za x %storepath%\%patch% -y -o”%storepath%”
echo.

We name some variables in case we ever change the paths then all the settings will be changed by just changing one line, we map a drive, I called it “x:”, you can call it whatever you want, we use wget to download the patch to the storepath, we unmap the drive…I think I had to use a mapped drive because weget didn’t like UNC paths , we then unzip the update to our “updates” directory.

This isn’t “fullproof” because I haven’t used this process in a while.  We had some machines (tablets) that went through a proxy server and we didn’t want to allow these tables any Internet except the specific SAAS (Software as a service) 3rd party vendor we were using.  So we made the server get the updates and each client to look at the server for updates.

There does look like there are registry settings for changing WHERE MSE gets udpates from, the default is “MMPC”.  I haven’t tried tweaking the setting since the clients I was putting it on couldn’t get to the site, they’d just update on user logon from a UNC path.  Next shift of users = new update!

Again; the scripts worked for me but there could be some tweaking.

Download wget

Download 7-zip

 

In my opinion Microsoft Security Essentials does a decent job for the price :p – they don’t just do spyware anymore, they also do malware and  antivirus!  For a smaller company (or even a larger one for that matter) looking to save money they can try using it rather than a more “corporate” package from the big vendors (Norton, McAfee, AVG, Panda, Avast, etc, etc, etc…the list goes on).  There might be just a few small problems; MSE doesn’t really have a central administration tool (or server) but that’s not a big deal with a small amount of work you can kind of make one or centrally manage it yourself.  It also doesn’t alert and administrator if it finds something, it only alerts the end user so expect a phone call from time to time.  Those are the biggest downfalls but depending on what you’re looking for or can tolerate it might work out for you.  I’ve distributed this on about 75 workstations and I haven’t really heard a peep from anyone.

Again though; it’s free and does a decent job so if you’re willing to ‘trust’ it and you’ve got other layers of protection (like not letting end users run as power user or local admin, local firewall, corporate firewall, strong passwords, encryption, etc, etc, etc) then your comfort level should be better (mine is).

So here’s the recipe for pushing out various version of MSE and configuring it with the options you want.  In our MSE batch of goodness you need these tools:

1. A Windows Domain (Could work without this but it’s helpful to have it)
2. Group Policy Know How (moderate)
3. Batch file skills (moderate)
4. A file share
5. 7-zip to extract the exe’s from M$
6. Install MSE so you can configure it then export your configuration via the registry

Download 7-zip and install it (link above)

Download MSE

Available Versions:

• Windows XP 32-bit
• Windows Vista/Windows 7 32-bit
• Windows Vista/Windows 7 64-bit

You should be downloading a file called:  mseinstall.exe <- Same name for all version but the packages are different

With 7-zip installed you should then be able to right click mseinstall.exe and extract the contents.

What I would do here is create a share folder “mse”, of course permissions are up to you but read only is fine for everyone, you could / should also make it a hidden share or ONLY a hidden share .  You can append a $ sign at the end of your share name like “mse$”, this way it won’t be browsable on your network.  You’ll have to know it’s there to find it (\\servername\mse$).  Copy your extracted folder to the mse share, maybe put an “xp” folder in the mse folder and then 7×32 and 7×64 folder to keep your versions straight if you need to deal with multiple versions.

For this example I downloaded the XP 32 bit version, I extracted it, I made my mse share with the appropriate subfolder named xp.

start > run > \\myserver\mse\xp or \\myserver\mse$\xp if you used a hidden share (recommended)

Folder contents are:

en-us <-Folder
x86 <- Folder
compappsconent.dll
epplauncher.exe
eppmanifest.dll
setup.ini
setupres.dll

* I just noticed this but in the setup.ini if you’re upgrading versions it looks like you can mark “ShowUpgradeEULA=true” to =false so you don’t have to agree to the EULA again

So what we’ve done so far is gather our ingredients, the tools and files we need to make our batch of goodness   We’ve downloaded and installed 7-zip, we’ve downloaded our versions of MSE and we’ve created our share then extracted the files to their respective folders.   Now it’s time to create out batch file that will:

1. Detect the OS
2. Detect the OS Architecture (32 or 64 bit)
3. Call the appropriate installer

In the \\servername\mse folder on your network create a batch file, I called mine push_mse.bat.  Then right click and ‘edit’ the file, it should open notepad.  For you sophisticated types you might like using notepad++, a favorite editing tool of mine because it highlights things in colors and has advanced search / replace filters.

In our batch file start by typing in or copy and pasting in this text with edits to match your environment:

****

@echo off

:: This is a comment – this will install MSE

echo Finding your Windows Version
:: Error = 0 for true
:: Error = 1 for false

:: This version number = Windows XP
ver | findstr/C:”Version 5.1.2600″
if %errorlevel% EQU 0 goto XP

:: This version number = Windows 7
ver | findstr/C:”Version 6.1.7601″
if %errorlevel% EQU 0 goto arch

:XP
:: Checking to see if the MSE folder exists, if it doesn’t then it’s not installed
if  exist “%ProgramFiles%\Microsoft Security Client” goto end
\\servername\mse\xp\x86\setup.exe /s /runwgacheck /o
reg import \\servername\mse\windows_security_essentials_default_settings.reg
goto end

:arch
:: Detecting 32 bit or 64 bit
wmic OS get OSArchitecture | findstr /C:”32-bit”
IF %ERRORLEVEL% EQU 0 goto 7-32
IF %ERRORLEVEL% GTR 0 goto 7_64

:7-32
:: Checking to see if the MSE folder exists, if it doesn’t then it’s not installed
if exist “%ProgramFiles%\Microsoft Security Client” goto end
\\servername\mse\7×32\x86\setup.exe /s /runwgacheck /o
reg import \\servername\mse\windows_security_essentials_default_settings.reg
goto end

:7-64
:: Checking to see if the MSE folder exists, if it doesn’t then it’s not installed
if  exist “%ProgramFiles%\Microsoft Security Client” goto end
\\servername\mse\7×64\amd64\setup.exe /s /runwgacheck /o
reg import \\servername\mse\windows_security_essentials_default_settings.reg
goto end

:end

****

I think that’s about right, the above is untested really, it’s similar to what I’m using but mine is much messier :p.  What you can do to see your errors is comment out the “@echo off” section and it’ll print every line on the screen, then end your script with a pause or put a bunch of pause statements in the script, I often do that to “step through” a program and when I’m done I get rid of all the pausing.  You can also “echo %errorlevel%” to see the output / result.

The ‘cherry on top’ are the settings.  You’ll notice above I’ve got a registry import for the settings.  To get these you’ll need to install MSE, configure it in the settings then export your settings from the registry to a reg file.  The settings seem to be the same from 32 bit to 64 bit and they have the same registry entry options.

start (or Windows Button how lame!) > regedit > OK or enter if you’re on Vista or 7.  This will open the registry editing tool, navigate down to:  HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Antimalware

I have only exported things like my Exclusions list / paths, the scan settings like allowing pause located under “Scan” and not joining of SpyNet under “SpyNet”.  Here’s a sample of my registry:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Antimalware\Exclusions\Paths]
“%programfiles%\\someprogramfilderhere”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Antimalware\Scan]
“AllowPause”=dword:00000001
“DisableCatchupQuickScan”=dword:00000001
“DisableCatchupFullScan”=dword:00000001
“CheckForSignaturesBeforeRunningScan”=dword:00000001
“QuickScanInterval”=dword:00000000
“ScheduleDay”=dword:00000000
“AvgCPULoadFactor”=dword:0000000a
“ScheduleTime”=dword:0000021c
“DisableRemovableDriveScanning”=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Antimalware\SpyNet]
“SpyNetReporting”=dword:00000000

* Recent update / addition here; a fee command line options I’ve used to do scans remotely with a tool like psexec:

psexec -u domain\username \\computername
It’ll prompt you for your password (must have admin privilages)
Then cd to your directory “cd c:\program files\Microsoft Security Client\Antimalware”
mpcmdrun -SignatureUpdate
* That command above updates the MSSE detection database
mpcmdrun -Scan -ScanType 2
* That command above will run a full system scan, 0 = default, 1 = quick scan, 2 = full system scan and 3 = a single file custom scan but you’ll need to point it at your file with “-File “c:\somehwere\somefile.extension”

You could probably export the entire “Microsoft Antimalware” hive and call it good too .  You’ll notice things like “ScheduleTime” – it’s a bit cryptic so if you want to adjust all the scanning times (mine is at noon I think) you can make the change on your local machine, check that key for the change then update your registry file.  Now what you want to do (and I’m not going to go into the how) is make a system startup group policy and point your new batch file in the system startup for PC’s in your organization.  You can do it many ways.  Something I’ve done in the past is have one system batch file that runs at startup and I tweak it for installs or settings like this.  Then you don’t have to create more group policies and have a plethora of batch files out there.  You just edit the one file and you’re good to go; that us…unless you need specialized settings for various departments or people.

Lastly, when the installation ‘kicks off’ there’s a bit of a pause or lul, it might take a few minutes to “do something” and for you to notice the Security Essentials little blue home logo in the system tray, the icon will probably be orange but it will shortly turn to blue after it updates which our command above on the install tells it to do .

There’s also a process I have to do an “update server” so all your clients don’t go out to the net for updates but rather they get updates from your local share.  I’m not going to post it here unless someone wants that process too.

All in all it’s been smooth and I’ve had no complaints.  It’s also saved the company a few thousand dollars in licensing fees to one of the big AV companies.

I was getting a few odd rsync backup issues that I banged my head against for a good…mmmmmm, 4 hours maybe.  It was perplexing only because the errors (to me) were cryptic.  I’m posting this because I didn’t seem to find any clear resolution on other sites.  I also can’t stand when someone posts an error only to later post an “Oh I figured it out” or “Oh I fixed it” WITHOUT leaving any clue as to HOW they fixed it.

First – lets download a cool chmod utility that shows you in a meaningful visual way how chmod works and what it all means:  Classical Web Designs

* That tool was quite helpful in figuring out what chmod number I had by comparing the ‘chmod string’ it also outputs.

Here goes:

RSYNC error:  rsync: change_dir#1 failed: Permission denied (13)

* This usually means the directory on your server has the wrong permissions.  Note that on FreeNAS the default permissions seem to be 755 or rwxr-xr-x.  To fix simply run this command ‘chmod 755 whateverfolder’.  You can also run ‘chmod -R 755 whateverfolder’ keep in mind though that I wouldn’t just yet because this will recurs through your folder tree and if it’s only the top level folder having problems you might overwrite sub folder permissions unintentionally.

* Interpreting the error, ‘change_dir’ means that rsync can’t navigate through your remote server directory tree

RSYNC error:  rsync: cannot stat destination “/somefolder/somewhere/there”: Input/output error (5)

RSYNC error:   rsync: ERROR: cannot stat destination : Permission denied (13)

* This usually means that rsync can’t WRITE to the destination folder so for example, “/somefolder/somewhere/there”.  Again this is a permissions issue and you should check the “there” folder permissions and work your way up.  Apparently when rsync ‘stats’ your stuff it’s shorthand for writing a temp file in that directory to facilitate the transfer.  I don’t pretend to know the nuts and bolts of how it woks but notice this to diagnose the issue.

“destination” – this isn’t your source :p

cd /somefolder/somewhere/there

ls -la

If you don’t see “rwxr-xr-x” or “youruser:guest” or depending on your system you might see “root:root” you should visit our old buddy chmod again and simply do a:  chmod 755 there or chmod 755 /somefolder/somewhere/there.  FYI on FreeNAS it’s ‘youruser:guest’, I also recently realized you could do a “chown -r youruser:guest /somefolder/somewhere/there” and that fixes permissions allowing the ownership to go from root:root to the afore mentioned.  I think this is preferred to chmod’ing.

RSYNC error:  rsync: recv_generator: failed to stat “somefolder/somewhere/there”: Input/output error (5)

* This usually mean that rsync can’t write to the directory in your SOURCE folder.  For example if you use cygwin on a Windows machine and you’re backing up c:\temp ” cygwin ‘/cygdrive/c/temp’” then go into that directory in cygwin and check your permissions.

cd /cygdrive/c/temp

ls -la

you might see an owner of ????????? or something that doesn’t look right.  In my case the user I was using to run rsync and backup directories didn’t own the directory and couldn’t write files to the root of that directory (I think rsync calls it a ‘manifest’ (list of files to xfer)).  So I then looked at the permissions on the Windows side of life and retried the xfer.

What it looked like:

drwx——+ 1 administrator  Domain Users 0 Jun  3  2010 Folder1
d———+ 1 ????????       Domain Users 0 May 12 23:11 Folder2

Success!

Moral of the story it would seem with RSYNC is that permissions are everything and if you get some type of error it’s usually because rsync can’t hook onto a file or directory with the user you’re running it as.  Let me know if this helped you because I spent 4 hours of my life to save you 3 hours and 45 minutes of yours :p