Diff file1 file2

---

Many time we come across situation that we want exact diff between two files ..
There are various Linux command which we can use like

# diff
# cmp

EG:
# cat  1
1
2
3
4
5
6
# cat  2
2
3
6
7
8

### Using diff
# diff -iw 1 2
1d0
< 1
4,5d2
< 4
< 5
6a4,5
> 7
> 8

Expln:
1d0< 1
- Add "1" at line 1 of file 2
4,5d2
< 4
< 5
- Add  "4" at line 4 of file 2
- Add  "5" at line 5 of file 2
6a4,5
> 7
> 8
- Add  "7" at line 7 of file 2
- Add  "8" at line 8 of file 2

-- Actual count as line 0 - N


Here it's tough to found whats mean of above if you are using it first time, What we expect is I want only those line which are missing in file 2 .

So have written a small PerlScript to achieve so, it's possible with BashScript too, but to parse a large file BashScript will take more time .

Script : diff.pl

#!/usr/bin/perl
# Written By - Shirish
#### perl  diff.pl file1 file2  OUTPUT
#### Will give you exact diff between file1 and file2 in OUTPUT file
##### Only lines that are in file1 but missing in file2
#################
#open a, "<filea";
#open b, "<fileb";
$x = $ARGV[0];
$y = $ARGV[1];
$z = $ARGV[2];
open a, "< $x";
open b, "< $y";
open (OP, ">$z") || die ("Unable to create Report");
local $/;
my @a = split /\n/, <a>;
my @b = split /\n/, <b>;
my %b = map { $_ => 1 } @b; # Make hash of B
my @res = grep { !defined $b{$_} } @a; # Everything in A not in B
print OP join "\n", @res;
#print join "\n", @res;
print "\n";

## How to use

## What's in file 1 which are absent in file 2
# perl diff.pl 1 2 1-2 
# cat 1-2
1
4
5

## What's in file 2 which are absent in file 1
# perl diff.pl 2 1 2-1
# cat 2-1
7
8

---

> Shirish Shukla

difference between two file linux, unix, find diff of two files, command diff, perl script to find diff of two files, compare two files, linux, linuxva, shirish, shukla

ConverT Shell Script to C - Binary [executable ]

Xxxxxxxxxxxxxx ConverT Shell Script to C - Binary [executable ] xxxxxxxxxxxxxxX

Hi many times we required to convert our script to be executable might be any of  below reasons .

- We don't want to share our script code
- We want to set suid, guid  that we can't do with shell scripts
- Want to run script without any extension as ./script.sh OR sh script.sh etc...

Here an fantastic toll that will hep you to achieve so .

Download & Install

rpmfind
Red Hat Linux el3 i386
Red Hat Linux el3 x86_64
Red Hat Linux el2.1 i386
CentOS-shc index

Note: Install proper Linux/Unix flavor rpm and architecture else it will not going to work as expected

Download:

# cat /etc/redhat-release
Red Hat Enterprise Linux ES release 3 (Taroon)
# uname -i
i386

### So have downloaded right rpm for CentOS-6
# wget  http://pkgs.repoforge.org/shc/shc-3.8.6-1.el3.rf.i386.rpm

Install:

# cd   /rpms/shc/
# rpm -ivh shc-3.8.6-1.el3.rf.i386.rpm
warning: shc-3.8.6-1.el3.rf.i386.rpm: V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]
   1:shc                    ########################################### [100%]

Write your script

AM taking example of   expire-date.sh

Convert to executable c binary

# cd  /myscripts; ls -lrt expire-date.sh
-rwxr-xr-x 1 root root 599 Mar 20 18:25 expire-date.sh

## Conver to exe using shc

# shc  -f  expire-date.sh
# gcc -o getexpiredt expire-pwd-date.sh.x.c
# mv getexpiredt  /bin/            <<-- As per your requirement

## Now test 

# file getexpiredt
getexpiredt: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, dynamically linked (uses shared libs), not stripped

## Execute

# getexpiredt
Enter Password Date [ DD:MM:YYYY EG: 25:02:2012] Format
                                                     DD: 01
                                                     MM: 02
                                                     YY: 2012
YOUR Password Will Expire on : Tue 1-May-2012
##

Have a fun ... Now remove your C code and sh code .

# rm -fr expire-pwd-date.sh*

---

--Shirish Shukla

convert shell script to c, c binary executable,shc -f cmd.sh command,linux, linuxva, shirish, shukla

Script to calculate Expiry Date

Xxxxxxxxxxxxxxxxxx Script To Calculate Your Expiry Date xxxxxxxxxxxxxxxX

Most of the time we require to calculate when my password/licence will expire ... this below for what i required by all users to calculate when there password get expire ..on unix ..

You need to provide input as DD : MM : YYYY .

File: expire-date.sh

#!/bin/bash

### Calculate Expire Date

### By- Shirish Shukla

############## Hardcoded

ExpireDays=90

Hours=00;Mins=00;Secs=0;zxone=IST

##############

echo -en "Enter Password Date [ DD:MM:YYYY EG: 25:02:2012] Format \n"

echo -en "\t\t\t\t\t\t     DD: " ; read dtys;

echo -en "\t\t\t\t\t\t     MM: " ; read mms;

echo -en "\t\t\t\t\t\t     YY: " ; read yys ;

############## Logic

EXPDT=$(date --date="$yys-$mms-$dtys $Hours:$Mins:$Secs $zxone + $ExpireDays days")

echo -en "\E[33m \n\nYOUR Password Will Expire on : " ; echo -en "\E[36m$EXPDT" | awk  '{print $1 " "$3 "-" $2 "-" $NF}'

echo -en "\e[0m\n"

## How to USE

# sh  expire-date.sh

Enter Password Date [ DD:MM:YYYY EG: 25:02:2012] Format

                                                     DD: 02

                                                     MM: 02

                                                     YY: 2012

YOUR Password Will Expire on : Wed 2-May-2012

Now I need to convert it into Binary Like I don;t want to Give same as to my users ... I can help you with this with my below Page ..







### Convert shell script to C binary .exe (executable)


--Shirish Shukla

script to calculate expiry date,bash script to find password expire date, convert shell script to c, c binary executable,shc -f cmd.sh command,linux, linuxva, shirish, shukla

backup using script imp files linux

Many time we come across situation to take a backup of file daily/weekly or sometime every hour .

Yes it may be that your filesystem are auto backuped by your backup tools as Snapshot/Veritas netbackup etc.. but still we always come across situation to that we want our some critical file (/etc/passwd, /etc/httpd/conf/httpd.conf etc..)  that had changed 1 hour back .

Below a small and simple script that can help us a way .. just schedule this in your crontab .. as per your requirement .

## MyBackup.sh

#!/bin/bash
#Shirish Shukla
#######################################
## Files to be backuped
Sources="/etc/passwd, /etc/group, /etc/shadow"
## Backup Save at Target:
Targetpath=/usr/Backup
#######################################
if [ ! -d $Target-path ]
then
    mkdir -p $Targetpath
fi
#######################################
create=`date +%a-%d-%b-%Y-AT:%H:%M`
delete=`date --date '1 month ago' +%a-%d-%b-%Y`
#######################################
for fles in `echo $Sources`
do
   fle=`echo $fles | sed  "s/,//g"`
    if [ ! -e $fle ]
    then
          echo "file: $fle not EXIST"
    fi
    lst=` echo $fle | awk -F"/" '{print $NF}'`
    tar -jcf $Targetpath/$lst-$create.bz2 $fle
done
#######################################
## Now delete 1 month old backups if exists any
rm -fr  $Targetpath/*$delete*
#######################################

## Check Backup

# ls -lrt  /usr/Backup/
-rw-r--r-- 1 root root 500 Feb 11 11:47 shadow-Sat-11-Feb-2012-AT:11:47.bz2
-rw-r--r-- 1 root root 938 Feb 11 11:47 passwd-Sat-11-Feb-2012-AT:11:47.bz2
-rw-r--r-- 1 root root 500 Feb 11 11:47 group-Sat-11-Feb-2012-AT:11:47.bz2

## Schedule crontab to take it 2 hour

# chmod 755 /root/MyBackup.sh
# crontab -e
01  */2 * * *  /root/MyBackup.sh

---

> Shirish Shukla

ssh without password

xxxxxxxxxxxxxxxxxxxxx SSH Login Without Password By- Shirish Shukla xxxxxxxxxxxxxxxxxxxxxxx

Example: ssh from shirish@client-1  to shukla@server-1  without password
Server-1: 192.168.8.10               
Client-1: 192.168.8.20

#####ON Client-1: 192.168.8.20 : user  shirish
### Login as shirish

# ssh-keygen -t dsa
----------------------------------------------->> Just press Enter don't type anything

Generating public/private dsa key pair.
Enter file in which to save the key (/home/shirish/.ssh/id_dsa):  <--Enter
Created directory '/home/shirish/.ssh'.                                    
Enter passphrase (empty for no passphrase):                          <--Enter
Enter same passphrase again:                                               <--Enter
Your identification has been saved in /home/shirish/.ssh/id_dsa.
Your public key has been saved in /home/shirish/.ssh/id_dsa.pub.
The key fingerprint is:
d8:6c:cb:2a:e7:24:01:43:9f:96:4a:45:e6:93:9b:f4 shirish@my.scratch.com

# ssh-copy-id -i .ssh/id_dsa.pub shukla@192.168.8.10

The authenticity of host '192.168.8.10 (192.168.8.10)' can't be established.
RSA key fingerprint is e8:0b:8a:d3:1a:d1:ce:ec:d8:f9:13:31:79:c8:03:ed.
Are you sure you want to continue connecting (yes/no)?  yes                        <--type yes
Warning: Permanently added '192.168.8.10' (RSA) to the list of known hosts.

Password:                                <-- type password of shukla on Server-1: 192.168.8.10               

Now try logging into the machine, with "ssh 'shukla@192.168.8.10'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

#####ON Server-1: 192.168.8.10
### Login as shukla  and confirm following
1> permission of /home/shukla/.ssh    700
$ ls -lrthd .ssh/
drwx------ 2 shukla shukla 4.0K Mar 12 13:51 .ssh/

2> permission of /home/shukla/.ssh/authorized_keys    600
$ ls -lrth .ssh/
-rw------- 1 shukla shukla 614 Mar 12 13:51 authorized_keys

########### Now login as shirish on Client-1:192.168.8.20
## ssh shukla@192.168.8.10
------------------------------------------------------>> It will not prompt for password

# Faced any problem Feedback in above contact me
#===============================Scratch=============================#
# AND Many More .....................Linux is Endless                                                                       #
#=========================== Hope you Liked IT ========================#
#                                                                                                   SSH-I  -- BY Shirish Shukla   #
#                                                                                                                 RHC Engineer 2010 #
#                                                                                                            shirish.linux@gmail.com #
#                                                                                                           shirishlinux.blogspot.com #
#                                  "Give Respect To Time One Day At Right Time, Time Will Respect You" #
#=================================================================#
# TRy Hard theres nothing that are un-achievable by HARDdd-WORKkk                                    #
#=================================================================#

linux ssh without password, password linux ssh, ssh, password sshd, sshd without password, login without password linux, ssh to a system without password

Script to take MBR backup

xxxxxxxxxxxx BASH To Take MBR BAckup -- Shirish Shukla xxxxxxxxxxxxx

#!/bin/bash

hdds=$(echo /dev/[hs]d?)

for hdd in $hdds

do

      ## take only last name of dir

        base=$(basename $hdd)

      ##take MBR Backup 0-512 bytes

        dd if=$hdd of=/tmp/$base.mbr count=1 bs=512

 echo $hdd

done

change date and time linux

change date and time linux

============ Treating with date command =================

1> date 081811002007 [mmddhhmmyyyy]--To set the Linux clock to 11th-Aug-2007-11-AM
2> timeconfig --Set your TIMEZONE
3> hwclock --hc to sys--->-->To set the Linux[system]clock from the HW clock
4> hwclock --utc sys to hc -->To set the HWclock from the Linux[system] clock

touch -t yyyymmddhhmm filename

echo "DATE:" `date --date "today" +%d-%m-%Y`             <--DATE:20-12-2010
echo "DATE:" `date --date "yesterday" +%d-%m-%Y`
echo "DATE:" `date --date "+5 minutes ago" +%d-%m-%Y-%M` <--present  -5 min
echo "DATE:" `date --date "-5minutes ago" +%d-%m-%Y-%M`  <---Present +5 min
echo "DATE:" `date --date "1 month ago -5 minutes ago" +%d-%m-%Y-%M`
                                                         <--present -1 month +5 min

Some importnat journal use paramiters
%a  --> Mon
%b  --> Jan
%d  --> date 01..31      %e  --> 1..31
%H  --> hour 0..23)      %k  --> 1..23
%M  --> minute (00..59)
%S  --> Seconds(00..60)
%I  --> hour 00..12)     %m  --> 1-12
%Y  --> year 2010        %y  --> year 10(last 2 digit)
%p  --> AM-PM            %p  --> am-pm
%T  --> %H:%M:%S

=================================== More n more ==================
%%--->a literal %
%a--->localeâs abbreviated weekday name (e.g., Sun)
%A--->localeâs full weekday name (e.g., Sunday)
%b--->localeâs abbreviated month name (e.g., Jan)
%B--->localeâs full month name (e.g., January)
%c--->localeâs date and time (e.g., Thu Mar  3 23:05:25 2005)
%C--->century; like %Y, except omit last two digits (e.g., 21)
%d--->day of month (e.g, 01)
%D--->date; same as %m/%d/%y
%e--->day of month, space padded; same as %_d
%F--->full date; same as %Y-%m-%d
%g--->last two digits of year of ISO week number (see %G)
%G--->year of ISO week number (see %V); normally useful only with %V
%h--->same as %b
%H--->hour (00..23)
%I--->hour (01..12)
%j--->day of year (001..366)
%k--->hour ( 0..23)
%l--->hour ( 1..12)
%m--->month (01..12)
%M--->minute (00..59)
%n--->a newline
%N--->nanoseconds (000000000..999999999)
%p--->localeâs equivalent of either AM or PM; blank if not known
%P--->like %p, but lower case
%r--->localeâs 12-hour clock time (e.g., 11:11:04 PM)
%R--->24-hour hour and minute; same as %H:%M
%s--->seconds since 1970-01-01 00:00:00 UTC
%S--->second (00..60)
%t--->a tab
%T--->time; same as %H:%M:%S
%u--->day of week (1..7); 1 is Monday
%U--->week number of year, with Sunday as first day of week (00..53)
%V--->ISO week number, with Monday as first day of week (01..53)
%w--->day of week (0..6); 0 is Sunday
%W--->week number of year, with Monday as first day of week (00..53)
%x--->localeâs date representation (e.g., 12/31/99)
%X--->localeâs time representation (e.g., 23:13:48)
%y--->last two digits of year (00..99)
%Y--->year
%z--->+hhmm numeric timezone (e.g., -0400)
%:z   -> +hh:mm numeric timezone (e.g., -04:00)
%::z  -> +hh:mm:ss numeric time zone (e.g., -04:00:00)
%:::z -> numeric time zone with : to necessary precision (e.g., -04, +05:30)
%Z    -> alphabetic time zone abbreviation (e.g., EDT)

### Other usefull operation with date command
# date  --date="next Monday"
# date  --date="10 days ago"


#####  Yesterday
# date --date="yesterday"   --OR--
#  date --date="-1 day"


##### Tomorrow
# date --date="next day"      --OR--
# date --date="-1 days ago"
# date --date='tomorrow'

##### After n  Day/week/month/year
# date --date='101 day'
# date --date='101 week'
# date --date='101 month'
# date --date='101 year'

### Similarly Before n Day/week/month/year Ago
# date --date='101 day ago'
# date --date='101 week ago'
# date --date='101 month ago'
# date --date='101 year ago'

#####  10 hours ago
# date --date='10 hour ago'
##### After 10 hours
# date --date='10 hour'   --OR--
# date --date=-'10 hour ago'     <-- Same applicable for all
##### 10 minutes ago
# date --date='10 minutes ago'
##### 10 seconds ago
# date --date=-'10 seconds ago'

##### Print date after 1 day 5 hour
# date --date='1 day 5 hour'

# date --date='-1 day -5 hour ago'



# Faced any problem Feedback in above contact me
#============================Scratch==============================#
#                                                     AND Many More .....................Linux is Endless               #
#========================== Hope you Liked IT =======================#
#                                                                                          CMDS DATE-- BY Shirish Shukla #
#                                                                                                                RHC Engineer 2010 #
#                                                                                                           shirish.linux@gmail.com #
#                                                                                                         shirishlinux.blogspot.com #
#                          "Give Respect To Time One Day At Right Time, Time Will Respect You" #
#==================================================================#
#                          TRy Hard theres nothing that are un-achievable by HARDdd-WORKkk  #
#==================================================================#

date configuration linux, linux, commands, Linux commands, kernel Linux, Linux mantra, linux history command, magic, Linux web, yum server configuration Linux, cron, anacron

remove spaces fom filename

Having a space in the file name is never a good idea.
If you are in need to remove space from all file names
within your current directory you can use a following
command to do so:

ls | grep " " | while read -r f; do mv -i "$f" `echo $f | tr -d ' '`; done

In case that you wich to substitute space within a file
name to underscore ( or any other character ) use a following
command to do so:

ls | grep " " | while read -r f; do mv "$f" `echo $f | tr ' ' '_'`; done

How it works? ls and grep will feed while loop with all files within
a current working directory which contain a space in their file name.
In the body of the while loop we will next execute mv command a translate
it file destination with tr command. Make sure to keep -i option enabled
when using mv command to avoid accidentally overwrite files.

scripting sort hand

## Count the occurrences of a specific word in a .txt file in bash shell.

# tr -s ' ' '\n' < myfile.txt | grep -c 'searchword'
# cat mydr/myfiles-1-* |grep -c "searchtest"

## Find Days in months(eg apr:30 days)

# tmnth=$(cal 4 2010 | egrep -v '[A-Za-z]' | wc -w)  

## gunzip & untar in 1 cmd

# cat May-2010.tar.gz | gunzip -d | tar -xvf -               
##
# gzcat test123.tar.gz | tar -xvf -
##
# tar -zcvf   May-2010.tar.gz

## Check is your variable is numeric or not?

# cat  isnum.sh
#!/bin/bash
# Script to test variable is numeric or not
# Shirish Shukla
# Pass arg1 as number
a1=$1
a=$(echo "$a1" |awk '{if($1 > 0) print $1; else print $1"*-1"}'| bc)
b=$(echo "scale=2;$a2/$a2 + 1" | bc -l 2>/dev/null)
if [[ $b > 1 ]]
then
    echo "$a1 is Numeric"
else
    echo "$a1 is Non Numeric"
fi

## Output
# sh isnum.sh 12
12 is Non Numeric

# sh isnum.sh 12-5+4
12-5+4 is Non Numeric

# sh isnum.sh abc
abc is Non Numeric

# sh isnum.sh shirish-shukla
shirish-shukla is Non Numeric

IFS bash variable

The IFS internal variable
One way around this problem is to change Bash's internal
IFS (Internal Field Separator) variable so that it splits
fields by something other than the default whitespace
(space, tab, newline), in this case, a comma.

eg:
#!/bin/bash
IFS=$',' # seperate input by comma
vals='/mnt,/var/lib/vmware/Virtual Machines,/dev,/proc,/sys,/tmp,/usr/portage,/var/tmp'
for i in $vals; do echo $i; done
unset IFS

with unset IFS, so that it returns to its default value.
This will avoid any potential problems that could arise
in the rest of your script.


$IFS (internal field separator) : This variable determines how Bash recognizes fields,
or word boundaries, when it interprets character strings.
$IFS defaults to whitespace (space, tab, and newline), but may be changed, for example,
to parse a comma-separated data file. Note that $* uses the first character held in $IFS.


Fields seperator:
echo $datapath | awk -F "/" '{print $NF}'|awk 'BEGIN { FS="[- . ]" } {print $1}'

The logic is that the intialisation
BEGIN{FS="[ ]|[;]|[,]"} ; OFS=","}
sets up the Field Separator variable FS as a regular expression which matches any one of " " ";" ","
and the Output Field Separator OFS to be ",".

 

my scripts

==========> Backup Script

#!/bin/bash
# Shirish Shukla backup script
# source-> /usr/rnd/*
# target-> /shirishva/backup/abcd-...
#
#------------ Backup string create: abcd-Wed-Oct-27-22:31 format

create=$(echo abcd-`date +%a-%b-%d-%H:%M`)

tar -jcf /shirishva/backup/$create.bz2 /usr/rnd/*

#------------------- delete 3 Hour old backup data del->string

t1=$(date --date='3 hour ago')

t2=$(echo $t1 | sed -e "s/[:]/ /g")

array=( `echo $t2` )

del=$(echo abcd-${array[0]}-${array[1]}-${array[2]}-${array[3]}:${array[4]})

rm -fr /shirishva/backup/$del.bz2

#---------------------------------

Text editors

In this chapter, we will discuss the importance of mastering an editor. We will focus mainly on the Improved vi editor.
After finishing this chapter, you will be able to:

• Open and close files in text mode
  
• Edit files
  
• Search text
  
• Undo errors
  
• Merge files
  
• Recover lost files
  
• Find a program or suite for office use

Vi(m)

Vim stands for "Vi IMproved". It used to be "Vi IMitation", but there are so many improvements that a name change was appropriate. Vim is a text editor which includes almost all the commands from the UNIX program vi and a lot of new ones.
Commands in the vi editor are entered using only the keyboard, which has the advantage that you can keep your fingers on the keyboard and your eyes on the screen, rather than moving your arm repeatedly to the mouse. For those who want it, mouse support and a GUI version with scrollbars and menus can be activated.
We will refer to vi or vim throughout this book for editing files, while you are of course free to use the editor of your choice. However, we recommend to at least get the vi basics in the fingers, because it is the standard text editor on almost all UNIX systems, while emacs can be an optional package. There may be small differences between different computers and terminals, but the main point is that if you can work with vi, you can survive on any UNIX system.
Apart from the vim command, the vIm packages may also provide gvim, the Gnome version of vim. Beginning users might find this easier to use, because the menus offer help when you forgot or don't know how to perform a particular editing task using the standard vim commands.

Using the Vim editor

Two modes

The vi editor is a very powerful tool and has a very extensive built-in manual, which you can activate using the :help command when the program is started (instead of using man or info, which don't contain nearly as much information). We will only discuss the very basics here to get you started.
What makes vi confusing to the beginner is that it can operate in two modes: command mode and insert mode. The editor always starts in command mode. Commands move you through the text, search, replace, mark blocks and perform other editing tasks, and some of them switch the editor to insert mode.
This means that each key has not one, but likely two meanings: it can either represent a command for the editor when in command mode, or a character that you want in a text when in insert mode.

Pronunciation

It's pronounced "vee-eye". Commands that switch the editor to insert mode: a will append: it moves the cursor one position to the right before switching to insert mode i will insert o will insert a blank line under the current cursor position and move the cursor to that line. Pressing the Esc key switches back to command mode. If you're not sure what mode you're in because you use a really old version of vi that doesn't display an "INSERT" message, type Esc and you'll be sure to return to command mode. It is possible that the system gives a little alert when you are already in command mode when hitting Esc, by beeping or giving a visual bell (a flash on the screen). This is normal behavior.

Basic commands

 Moving through the text is usually possible with the arrow keys. If not, try:

• h to move the cursor to the left
  
• l to move it to the right
  
• k to move up
  
• j to move down

SHIFT-G will put the prompt at the end of the document.
ctl+s -- stop   ctl+q --resume

Basic operations
These are some popular vi commands:

• n dd will delete n lines starting from the current cursor position.
  
• n dw will delete n words at the right side of the cursor.
  
• x will delete the character on which the cursor is positioned
  
• :n moves to line n of the file.
  
• :w will save (write) the file
  
• :q will exit the editor.
  
• :q! forces the exit when you want to quit a file containing unsaved changes.
  
• :wq will save and exit
  
• :w newfile will save the text to newfile.
  
• :wq! overrides read-only permission (if you have the permission to override permissions, for instance when you are using the root account.
  
• /astring will search the string in the file and position the cursor on the first match below its position.
  
• / will perform the same search again, moving the cursor to the next match.
  
• :1, $s/word/anotherword/g will replace word with anotherword throughout the file.
  
• yy will copy a block of text.
  
• n p will paste it n times.
  
• :recover will recover a file after an unexpected interruption.

Hope you Like it:        

  
                                                                                   29 aug 2010
                                                                              --Shirish Shukla