Create filesystem in a partition

One of my application creates un-necessary logs or dumps that fills my filesystem .
Say example I have only root partition "/" and other filesystems(/var/, /usr etc..) are in "/" .
Sometime one of my application creates lots of files and unwaned logs at path /var/log/mycore and that fills my root partition and my oher application stops working .

I want to give a permanent solution, but I have some limitations as below
- I don't have extra free disk block to create new partition
- I can't add any any new disk/LUN etc .
- I can't stop that application logging etc...

Here's solution that have applied on one of my server and it's working as am expected .

## Example

# df -h  /
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              72G   57G   12G  79% /

## I have 12 GB free space now check size of dir: /var/log/mycore

# du -sch /var/log/mycore
2G     /var/log/mycore
2G     total

## Okay now we have 2 way
1> Define quota .. but not preferred .
2> Create a filesystem with allocated space

## Am preferring to go with way-2 as below
As have 12G free space on "/", so I will allocate 3GB to /var/log/mycore as below

Step-1: Create a file usign dd with size 3GB
# dd  if=/dev/zero of=/var/log/FS_mycore bs=1024M count=3

Step-2: Create file system as per your requirement here am going with ext4
# mkfs.ext4  /var/log/FS_mycore

Step-3: Stop your application

Step-4:  Move /var/log/mycore  and create a new directory (Note: Reserve the permissions as original dir has)
# mv /var/log/mycore  /var/log/mycore_old
# mkdir  /var/log/mycore

Step-5: Mount and copy back all files and delete /var/log/mycore_old
# mount  -o loop  /var/log/FS_mycore  /var/log/mycore
# mount  -l


# df -h  /var/log/mycore/
Filesystem            Size  Used Avail Use% Mounted on
/var/log/FS_mycore    3.0G   68M  2.8G   2% /var/log/mycore/

# cp -rp  /var/log/mycore_old/* /var/log/mycore/

# cd  /var/log/mycore/ ; ls -lrth
# rm -fr  /var/log/mycore_old/

## Make entry in fstab
/var/log/mycore_old     /var/log/mycore         ext4    defaults,loop        0 0

##  Testing

# umount  /var/log/FS_mycore # mount -a
# mount -l 

## Step-6: Don't forget to start your application and check logs and all to verify all are okay ...

 

---

> Shirish Shukla

Diff file1 file2

---

Many time we come across situation that we want exact diff between two files ..
There are various Linux command which we can use like

# diff
# cmp

EG:
# cat  1
1
2
3
4
5
6
# cat  2
2
3
6
7
8

### Using diff
# diff -iw 1 2
1d0
< 1
4,5d2
< 4
< 5
6a4,5
> 7
> 8

Expln:
1d0< 1
- Add "1" at line 1 of file 2
4,5d2
< 4
< 5
- Add  "4" at line 4 of file 2
- Add  "5" at line 5 of file 2
6a4,5
> 7
> 8
- Add  "7" at line 7 of file 2
- Add  "8" at line 8 of file 2

-- Actual count as line 0 - N


Here it's tough to found whats mean of above if you are using it first time, What we expect is I want only those line which are missing in file 2 .

So have written a small PerlScript to achieve so, it's possible with BashScript too, but to parse a large file BashScript will take more time .

Script : diff.pl

#!/usr/bin/perl
# Written By - Shirish
#### perl  diff.pl file1 file2  OUTPUT
#### Will give you exact diff between file1 and file2 in OUTPUT file
##### Only lines that are in file1 but missing in file2
#################
#open a, "<filea";
#open b, "<fileb";
$x = $ARGV[0];
$y = $ARGV[1];
$z = $ARGV[2];
open a, "< $x";
open b, "< $y";
open (OP, ">$z") || die ("Unable to create Report");
local $/;
my @a = split /\n/, <a>;
my @b = split /\n/, <b>;
my %b = map { $_ => 1 } @b; # Make hash of B
my @res = grep { !defined $b{$_} } @a; # Everything in A not in B
print OP join "\n", @res;
#print join "\n", @res;
print "\n";

## How to use

## What's in file 1 which are absent in file 2
# perl diff.pl 1 2 1-2 
# cat 1-2
1
4
5

## What's in file 2 which are absent in file 1
# perl diff.pl 2 1 2-1
# cat 2-1
7
8

---

> Shirish Shukla

difference between two file linux, unix, find diff of two files, command diff, perl script to find diff of two files, compare two files, linux, linuxva, shirish, shukla

ConverT Shell Script to C - Binary [executable ]

Xxxxxxxxxxxxxx ConverT Shell Script to C - Binary [executable ] xxxxxxxxxxxxxxX

Hi many times we required to convert our script to be executable might be any of  below reasons .

- We don't want to share our script code
- We want to set suid, guid  that we can't do with shell scripts
- Want to run script without any extension as ./script.sh OR sh script.sh etc...

Here an fantastic toll that will hep you to achieve so .

Download & Install

rpmfind
Red Hat Linux el3 i386
Red Hat Linux el3 x86_64
Red Hat Linux el2.1 i386
CentOS-shc index

Note: Install proper Linux/Unix flavor rpm and architecture else it will not going to work as expected

Download:

# cat /etc/redhat-release
Red Hat Enterprise Linux ES release 3 (Taroon)
# uname -i
i386

### So have downloaded right rpm for CentOS-6
# wget  http://pkgs.repoforge.org/shc/shc-3.8.6-1.el3.rf.i386.rpm

Install:

# cd   /rpms/shc/
# rpm -ivh shc-3.8.6-1.el3.rf.i386.rpm
warning: shc-3.8.6-1.el3.rf.i386.rpm: V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]
   1:shc                    ########################################### [100%]

Write your script

AM taking example of   expire-date.sh

Convert to executable c binary

# cd  /myscripts; ls -lrt expire-date.sh
-rwxr-xr-x 1 root root 599 Mar 20 18:25 expire-date.sh

## Conver to exe using shc

# shc  -f  expire-date.sh
# gcc -o getexpiredt expire-pwd-date.sh.x.c
# mv getexpiredt  /bin/            <<-- As per your requirement

## Now test 

# file getexpiredt
getexpiredt: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, dynamically linked (uses shared libs), not stripped

## Execute

# getexpiredt
Enter Password Date [ DD:MM:YYYY EG: 25:02:2012] Format
                                                     DD: 01
                                                     MM: 02
                                                     YY: 2012
YOUR Password Will Expire on : Tue 1-May-2012
##

Have a fun ... Now remove your C code and sh code .

# rm -fr expire-pwd-date.sh*

---

--Shirish Shukla

convert shell script to c, c binary executable,shc -f cmd.sh command,linux, linuxva, shirish, shukla

Script to calculate Expiry Date

Xxxxxxxxxxxxxxxxxx Script To Calculate Your Expiry Date xxxxxxxxxxxxxxxX

Most of the time we require to calculate when my password/licence will expire ... this below for what i required by all users to calculate when there password get expire ..on unix ..

You need to provide input as DD : MM : YYYY .

File: expire-date.sh

#!/bin/bash

### Calculate Expire Date

### By- Shirish Shukla

############## Hardcoded

ExpireDays=90

Hours=00;Mins=00;Secs=0;zxone=IST

##############

echo -en "Enter Password Date [ DD:MM:YYYY EG: 25:02:2012] Format \n"

echo -en "\t\t\t\t\t\t     DD: " ; read dtys;

echo -en "\t\t\t\t\t\t     MM: " ; read mms;

echo -en "\t\t\t\t\t\t     YY: " ; read yys ;

############## Logic

EXPDT=$(date --date="$yys-$mms-$dtys $Hours:$Mins:$Secs $zxone + $ExpireDays days")

echo -en "\E[33m \n\nYOUR Password Will Expire on : " ; echo -en "\E[36m$EXPDT" | awk  '{print $1 " "$3 "-" $2 "-" $NF}'

echo -en "\e[0m\n"

## How to USE

# sh  expire-date.sh

Enter Password Date [ DD:MM:YYYY EG: 25:02:2012] Format

                                                     DD: 02

                                                     MM: 02

                                                     YY: 2012

YOUR Password Will Expire on : Wed 2-May-2012

Now I need to convert it into Binary Like I don;t want to Give same as to my users ... I can help you with this with my below Page ..







### Convert shell script to C binary .exe (executable)


--Shirish Shukla

script to calculate expiry date,bash script to find password expire date, convert shell script to c, c binary executable,shc -f cmd.sh command,linux, linuxva, shirish, shukla

HP-UX Basic Admin -3 - Software and imp log files

Xxxxxxxxxxxxxx HP-UX - Basic Administration - III  xxxxxxxxxxxxxxxxxxxxX
------------ Index -----------------
1> System Info
2> Basic Admins
3> Software and imp log files
4> Patching and Recovery          <Yet to update>
5> Backup and Tools                  <Yet to update>     
5> Advanced Administrations     <Yet to update>
------------------------------------

### Installed Softwares
# swlist -l product
# swlist -l patch
# swlist -l file
# swlist -l product | grep -i <ur-pack>

## verify package/patches
# swlist -l -a state
# swverify
# swlist -s   <path-of-your-pack>

### Install
# swinstall -s  <path-of-your-pack>
>>>> Select Package -> Go to Action > Install ...

## Varify is Installed
# swlist -l product | grep -i  <your-pack-name>


### Un-install
swremove
##

##### Important Files/log files etc..
### Log Files
# more  /var/adm/syslog/syslog.log
# more  /var/adm/syslog/mail.log
# more  /var/adm/cron/log
# more  /var/adm/syslog/syslog.log
# more /var/sam/log/samlog


--Shirish Shukla

hp-ux, hp unix, hp-ux basic admin commands, hp-ux getconf, hp-ux swinstall, software, log files , linux, linuxva, shirish, shukla

HP-UX Basic Admin -2 - Basic Admins

Xxxxxxxxxxxxxx HP-UX - Basic Administration - II  xxxxxxxxxxxxxxxxxxxxX
------------ Index -----------------
1> System Info
2> Basic Admins
3> Software and imp log files
4> Patching and Recovery          <Yet to update>
5> Backup and Tools                  <Yet to update>     
5> Advanced Administrations     <Yet to update>
------------------------------------


## Display Hostname
# hostname

## IP Address
# netstat -ani

## Disk Utilization
# bdf
# df -Pk

## Show run levels
# who -r
   .       run-level 3  Oct 28 11:22    3    0    S
[[ 0: shut down
s,S: single-user
1: sys admin
2: multiuser
3*: share NFS
4*: start GUI
5: user-defined
6: user-defined ]]

### Shutdown/Restart
<<HALT>>
# shutdown -h -y now  
# init 0                                          
# shutdown -hy 120    
# shutdown -h -y 120                   <<Shutdown in 120 sec>>
# shutdown -Fr now                     <Force fsck upon reboot SAFE way>

<<Reboot>>
# shutdown -r -y now 
# shutdown -r 0                              
# reboot 0

<<To shutdown to single user mod>>
# shutdown -y 0
# shutdown -y now
# shutdown 0
# init 1
# hpux -is
# init S
# reboot -h    <<Come from Single user Mode>>

### LAN IP Info
# lanscan        
# lanscan  -v   
# netstat -in   
# ifconfig lanx   [ x=0/1/2/....]

Note: lanscan is deprecated in 11.31 now can find  # nwmgr

### Assign/Display  IP
## GET
# ifconfig lan0
# netstat -ani  
#

## Set
# landiag          
# lanadmin              
# vi /etc/rc.config.d/netconf
HOSTNAME="Shirish"
INTERFACE_NAME[0]=lan0
IP_ADDRESS[0]=192.168.8.10
SUBNET_MASK[0]=255.255.255.0
BROADCAST_ADDRESS[0]=""
INTERFACE_STATE[0]=""
DHCP_ENABLE[0]=0
INTERFACE_MODULES[0]=""

##NOTE: Above required restart to take effects FOR immediate use below but note it's temp
# ifconfig lan3 192.168.8.10 netmask 255.255.255.0  up     

## Up network interface
# ifconfig lan3 up

## Down network interactive
# ifconfig lan3 down

## ARP/Routing Table info/Define new route
# arp -a
# netstat -ar
# route add default 192.168.8.1 1

## Start/Stop Network
/sbin/init.d/net stop; /sbin/init.d/net start
Note: There's NO restart option you have to stop; then start
WARNING: Pass both as single command as above else you will loose control of your server/system

### Hostname start/stop
# /sbin/init.d/hostname stop
/sbin/init.d/hostname start
   
## Set timezone
# vi /etc/TIMEZONE




--Shirish Shukla

hp-ux, hp unix, hp-ux basic admin commands, hp-ux getconf, linux, linuxva, shirish, shukla

HP-UX Basic Admin -1 - Sys Info

Xxxxxxxxxxxxxx HP-UX - Basic Administration - I xxxxxxxxxxxxxxxxxxxxX
------------ Index -----------------
1> System Info
2> Basic Admins
3> Software and imp log files
4> Patching and Recovery          <Yet to update>
5> Backup and Tools                  <Yet to update>     
5> Advanced Administrations     <Yet to update>
------------------------------------

## 64 or 32 bit ??
# getconf KERNEL_BITS
64

# file /stand/vmunix
/stand/vmunix:  ELF-64 executable object file - PA-RISC 2.0 (LP64)

# find / -name print_mani*
/opt/ignite/binpa/print_manifest
# /opt/ignite/binpa/./print_manifest | grep -i mod
    Model:              9000/800/L3000-8x
    OS mode:            64 bit

# getconf HW_CPU_SUPP_BITS
64

## Get detailed information about System
# find / -name print_mani*
/opt/ignite/binpa/print_manifest
# /opt/ignite/binpa/./print_manifest

## Get Model
# model
9000/800/L3000-7x
### Configuration files
# cd /etc/rc.config.d

### Your Kernel
# cd /stand/vmunix

### Display Kernel Paramiters
# kctune
# sysdef
# kmtune

## Get memory info
# /usr/sam/lbin/./getmem
# dmesg | grep -i Phy
# /opt/ignite/bin/print_manifest    <-- An detailed infor about your system

### All Kernel Parms and Hwrs
# /usr/sbin/sysdef or ioscan –fn

### Devices Messages
# dmesg

## Get System Serial Number
# echo "sel dev 1 ;info;wait;infolog" | cstm |grep -i "System Serial Number"
   System Serial Number...: SGH444039B

### Check swap space
# swapinfo
# swapinfo -m     <- IN MB
# swapinfo -tm    <- Disp Total Row


### Check attached Disk/devices/hws
# ioscan -funC disk
# ioscan -C disk
# ioscan -fnC lan
# ioscan -fnC tape   
# ioscan -fnC  lun

Here:
-u: Show usable devices i.e don;t rescan
-f: Full listing
-C: Show Class
-n: Device name

### Format a disk
# ioscan -funC disk
Class     I  H/W Path     Driver S/W State   H/W Type     Description
=====================================================================
disk      1  0/0/1/1.2.0  sdisk CLAIMED     DEVICE       HP 73.4GST373454LC
                         /dev/dsk/c1t2d0   /dev/rdsk/c1t2d0
..
..

# mediainit /dev/rdsk/c1t2d0

### Disk detailed info
# diskinfo -v /dev/rdsk/c1t2d0


--Shirish Shukla

hp-ux, hp unix, hp-ux basic admin commands, hp-ux getconf,hp-ux get 32 or 64 bit, disk info etc.., unix linux, linuxva, shirish, shukla

backup using script imp files linux

Many time we come across situation to take a backup of file daily/weekly or sometime every hour .

Yes it may be that your filesystem are auto backuped by your backup tools as Snapshot/Veritas netbackup etc.. but still we always come across situation to that we want our some critical file (/etc/passwd, /etc/httpd/conf/httpd.conf etc..)  that had changed 1 hour back .

Below a small and simple script that can help us a way .. just schedule this in your crontab .. as per your requirement .

## MyBackup.sh

#!/bin/bash
#Shirish Shukla
#######################################
## Files to be backuped
Sources="/etc/passwd, /etc/group, /etc/shadow"
## Backup Save at Target:
Targetpath=/usr/Backup
#######################################
if [ ! -d $Target-path ]
then
    mkdir -p $Targetpath
fi
#######################################
create=`date +%a-%d-%b-%Y-AT:%H:%M`
delete=`date --date '1 month ago' +%a-%d-%b-%Y`
#######################################
for fles in `echo $Sources`
do
   fle=`echo $fles | sed  "s/,//g"`
    if [ ! -e $fle ]
    then
          echo "file: $fle not EXIST"
    fi
    lst=` echo $fle | awk -F"/" '{print $NF}'`
    tar -jcf $Targetpath/$lst-$create.bz2 $fle
done
#######################################
## Now delete 1 month old backups if exists any
rm -fr  $Targetpath/*$delete*
#######################################

## Check Backup

# ls -lrt  /usr/Backup/
-rw-r--r-- 1 root root 500 Feb 11 11:47 shadow-Sat-11-Feb-2012-AT:11:47.bz2
-rw-r--r-- 1 root root 938 Feb 11 11:47 passwd-Sat-11-Feb-2012-AT:11:47.bz2
-rw-r--r-- 1 root root 500 Feb 11 11:47 group-Sat-11-Feb-2012-AT:11:47.bz2

## Schedule crontab to take it 2 hour

# chmod 755 /root/MyBackup.sh
# crontab -e
01  */2 * * *  /root/MyBackup.sh

---

> Shirish Shukla