Search This Blog

Loading...

Wednesday, December 31, 2008

Disabling mod_security 2 for individual directories

I ran into some trouble with mod_security falsely blocking my joomla administrator pages.

The old method of disabling mod_security by placing a SecFilterEngine Off SecFilterScanPOST Off in a .htaccess doesn't work anymore. This doesn't seem to be clear in the docs and I only found this info in a mod-sec mailing post.

So the rules now need to go directly into the apache conf files.

I have most sites hosted on a plesk server. In plesk you can add to the virtual host config inside a vhost.conf file found in "/var/www/vhosts/conf/vhost.conf".
Create or edit this file and enter:

SecRuleEngine Off

Apply the vhost changes if a new vhost file with:
/usr/local/psa/admin/bin/websrvmng -v -a

And reload the httpd server with:

service httpd restart

Better still disable specific rules with
SecRuleRemoveById _rule_id_

Instead of the sledgehammer SecRuleEngine Off

What & Where to Backup


Backing up the information on your hard drive on a regular basis is critical for anyone who uses a computer routinely. You have spent hours-on-end recording the information that is on your hard drive, and much of that data is extremely important and irreplaceable stuff! You should be routinely backing up at least your new data files on a weekly basis. "Data" files include databases, documents, spreadsheets, etc. Anything YOU created on your hard drive is a data file.

If you don't have a second hard drive or network attached drive available to you, your choices include a CD burner (CD-RW drive) or a DVD burner (DVD-RW drive). Eons ago, the only way to perform a backup on a home computer was to save the data onto floppy disks. A typical PC's floppy disk holds only 1.44 MB of data, so you might get one or two files on a floppy disk. Today, if you have a file that is larger than 1.44 MB (like a moderate OpenOffice Impress presentation file, or a Writer document file with graphics) an ordinary floppy disk has no hope of holding even one whole file. CD-R or CD-RW disks hold much more than floppy disks and are your best choice for routine storage of information on removable media. A typical recordable CD holds between 450 and 500 times what will fit on a single floppy disk. DVDs are significantly larger than that.


When backing-up data files, it is not necessary to backup every file you have on your hard drive. (Although it's a good idea to do so monthly. See below.) Many of you listeners to the Going Linux podcast are Linux users. For you, few files (if any) outside of your Home directory need to be backed-up at all. If a disaster were to happen, and you lost your computer or your hard drive, you could simply reinstall the Linux operating system and restore the backed-up contents of your Home directory. Your Home directory contains the information that you have created yourself, including your mail file, and the settings and preferences for most of your applications. In Episodes
29 and 31 of the podcast, we discuss specific Linux applications you can use to accomplish your backups.

If you have a home network you may be lucky enough to own a multi-gigabyte network-attached hard drive. Instead of being stored on your computer's hard drive, the files you copy to the networked drive are stored safely off of your computer. That way, if something happens to your computer's hard drive, or if you have your laptop stolen, all your files are safely stored on an
external device. (In the illustration below, I am using Kubuntu's Dolphin file manager to take my entire "home" directory and copy it to an external 500GB drive, shared on my network using SMB.)
Backing up to an external drive


An equivalent to a network drive (well, sort of...) would be an online storage service like Xdrive or Box.net. The advantage to this kind of backup is that your files are stored on the Internet. Even if the worst were to happen, and you lost everything to a fire or flood, your backups would be safe.



Cheap Disaster Recovery Insurance
The easiest way to backup inexpensively, is to get a firewire or USB 2.0 external hard drive and copy everything in your Home directory to it on a regular basis. If you don't have an external hard drive, you can use CD/DVD Burning software like K3B, and a CD-RW or DVD-R drive, to write to CD-R or DVD-R discs. These discs are very cheap, but you can only write to them. You can't erase the contents of a CR-R or DVD-R and re-use the disc. Re-writable CD and DVD media, on the other hand, have become very affordable in recent years. You can read, erase and write to CD-RW and DVD-RW disks much like you can with floppy disks. They just take a while to format the first time you use them.
Backup to CD using K3B or other burning software


At least monthly, you should make a complete backup of everything important on your hard drive -- using backup software, or by simply copying to a disk, a second hard drive or a network drive. Linux applications such as
rsync and Simple Backup can automate these backups for you. You do this so that if your computer or hard drive were to be stolen, destroyed or damaged beyond repair, you would have a "disaster recovery" copy. From a "disaster recovery" copy, you could restore your entire Home directory to the condition it was in, at the time of your last backup. If you do such a complete backup monthly, the worst case is that you have to restore only 1 month's worth of data from your incremental data backups. (See below.)


How Often to Backup
In reality, how often you should backup depends on several factors. In general, the more you use your computer or the more critical the work you are doing, the more often you should back-up.

External hard drives have become so inexpensive that it is wise to include an external hard drive in your budget when you are buying a new computer. Use the external drive to make a complete backup of your entire hard drive every month. You can use many of the backup utilities available in your Linux distribution's package repositories to automate this process.
Clonezilla Live is such a utility. You can even setup some of this software so that it copies only new files, and updates new versions of files on your computer, to the backup device, never deleting anything from the copy. What a lifesaver that can be!

Today most computers come equipped with at least a CR-RW drive. Invest in a set of CD-RW disks, or even just CD-R disks. So if you don't have a second hard drive, use the CD-RW drive to make the monthly full backup. Each week you should back up your critical data files in an incremental backup (backup only what has changed since your last full backup) onto a recordable or re-writable CD.


Don't wait. Do it now -- or you will wish you had!

Using Clonezilla Live to Backup your Computer

Using Clonezilla Live to Backup your Computer

In Episodes 29 and 31 of the podcast, we discuss specific applications you can use to backup the data on your computer. We mentioned Clonezilla Live as one of those applications, but this is not your average Linux backup utility. It is a Live CD (you boot from the CD to use it) that can make backups of your entire hard drive, including your Linux AND Windows partitions. This means that whether you have a single operating system on yoru hard drive, or if you dual-boot your computer, you can make a complete backup of all your partitions in a single step. In addition, you can compress the final backup image so that if you are backing up a 250 GB hard drive that is 50% full (it has 125 GB of free space) the backup will not take a full 250 GB of space on the backup drive.


Using Clonezilla Live
Here is the step-by-step on making a backup of a 40 GB hard drive onto an external USB drive. Unlike some other backup programs, Clonezilla does not ask you whether you are making a backup or restoring from one until almost the end of the menu selections. As a result, although these instructions are for making a backup to an image file, they are almost identical for the restoration process. Note that since the menus of selections are presented in text screens, you will be using the tab key and the arrow keys to move between selections, the spacebar to mark options, and the Enter key to accept your choices and move to the next screen.
  1. Boot your computer from the Clonezilla Live CD.
  2. At the boot screen, select the default.
    Clonezilla Live boot screen
  3. Allow all the text to scroll past the screen as Clonezilla determines your hardware configuration and completes the bootup process.
  4. Choose a language. English and Traditional Chinese are your choices.
  5. Select a keyboard layout if you use a non-standard keyboard, otherwise the default will work fine. Choose to start Clonezilla.
  6. You can choose to make a disk-to-disk backup or a diks to image backup. I chose:
    device-image disk/partition to/from image
  7. To make your backup to an external device like a USB drive, select:
    local_dev Mount local device
  8. You are then prompted to insert usb device.
  9. In my case, the screen showed two hard drives. Cursor to the one that represents the destination for your backup files. Use the spacebar to make your selection. Here are the selections that appeared on my screen:
    ( ) hda1 (a long description of my hard drive)
    (*) sda1 (a description of my external USB 2.0 drive)
  10. Mounting an external USB drive could take a long time, depending on such factors as your computer's processor speed and RAM, and the size of the USB drive, so BE PATIENT! When prompted, press .
  11. The next screen is where you indicate whether you want to make a backup or restore from one, and whether yu want to save the entire disk or individual partitions. Your choices are:
    savedisk Save_client_entire disk
    restoredisk Restore _client_entire_disk
    saveparts Save_client_partitions
    restoreparts Restore_client_partitions
    exit
    I chose the first option.
  12. At the priority menu, you can choose which programs Clonzilla will use, and in which order. Your choices are:
    -q Priority: ntfsclone > partimage > dd
    -q1 Priority: Only dd (support all filesystem, but inefficient)
    -q2 Priority: ntfsclone, parclone (experimental) > partimage > dd
    Priority: partimage > dd (no ntfsclone)
    The first choice is the best in most cases. It tries ntfsclone to make a backup of any NTFS (Windows) partition, then partimage and dd to backup any Linux partitions.
  13. The next screen shows advanced extra parameters. Only the first option is selected by default, and that is the way I left it.
    [*] -c Client waits for confirmation before cloning
  14. Next, you have a choice of compression settings. The first one worked fine for me.
    -z1 Use gzip compressionn (fast but smaller image)
    -z2 Use bzip2 compression (slowest but smallest image)
    -z3 Use lzo compression (faster, image size ~ gzip)
    -z0 without compression (fastest but largest image)
  15. You are prompted to give the image a name. The default name indicates the date of the backup.
  16. Next, select the disk to backup. In my case there was only one choice:
    [*] hda disk description
  17. Clonezilla then provides a summary of what is being backed-up, and where the backup will be located. Make sure that you have chosen the right
  18. Finally, you are asked one final question:
    Are you sure you want to continue ? (y/N)
The remaining step is to sit back and wait for your backup to be completed.

HowtoForge has created an excellent, illustrated article on how to back up and restore hard drives using CloneZilla Live. Click here to see the tutorial.

Video using a recovery CD to restore an image






Introduction to Sun Solaris

Image
Hệ điều hành Solaris là sự lựa chọn tốt cho các ứng dụng doanh nghiệp. Trong các dòng sản phẩm Unix thương mại hoá thì Solaris là hệ điều hành tiên tiến nhất với số lượng cài đặt lớn nhất. Solaris thích hợp cả Sparc Platform lẫn x64/x86 Platform. Để xây dựng ra Sun Solaris 10, Sun Microsystems đã tiêu tốn hớn 500 triệu USD và 3.000 năm/công kỹ sư thiết kế.

Thiết kế cho các trung tâm dữ liệu hiện đại, Solaris 10 được xem là hệ điều hành Unix nhanh nhất mà Sun đã từng cung cấp với tốc độ nhanh hơn 40% so với Solaris 9 trong tốc độ xử lý của máy chủ Web trên cả SPARC và x86. Sun có kế hoạch phát triển Solaris 10 trên các hệ thống sử dụng bộ xử lý SPARC x86, AMD64, EM64T. Solaris 10 là nền tảng cho các hệ thống từ nhiều nhà cung cấp khác nhau như Dell, Fujitsu, Siemens, IBM, HP...
Hệ điều hành Solaris là sự lựa chọn tốt cho các ứng dụng doanh nghiệp. Trong các dòng sản phẩm Unix thương mại hoá thì Solaris là hệ điều hành tiên tiến nhất với số lượng cài đặt lớn nhất. Solaris thích hợp cả Sparc Platform lẫn x64/x86 Platform. Để xây dựng ra Sun Solaris 10, Sun Microsystems đã tiêu tốn hớn 500 triệu USD và 3.000 năm/công kỹ sư thiết kế.

Thiết kế cho các trung tâm dữ liệu hiện đại, Solaris 10 được xem là hệ điều hành Unix nhanh nhất mà Sun đã từng cung cấp với tốc độ nhanh hơn 40% so với Solaris 9 trong tốc độ xử lý của máy chủ Web trên cả SPARC và x86. Sun có kế hoạch phát triển Solaris 10 trên các hệ thống sử dụng bộ xử lý SPARC x86, AMD64, EM64T. Solaris 10 là nền tảng cho các hệ thống từ nhiều nhà cung cấp khác nhau như Dell, Fujitsu, Siemens, IBM, HP...

Solaris 10 có trên 600 đặc tính mới và là hệ điều hành Unix hỗ trợ nhiều nhà cung cấp  nhất, từ SPARC tới X86 và AMD Opteron.

Các dòng sản phẩm Oracle 10g (bao gồm Oracle Database 10g và Oracle Application Server 10g) cũng sẽ được hỗ trợ trên Sun Solaris 10 cho các máy chủ sử dụng bộ xử lý Intel Xeon, AMD Opteron và SPARC.

Theo Sun, Solaris 10 có hơn 600 cải tiến mới trong hệ thống file ZFS, các bộ chứa và khả năng tương hợp giữa Unix và Linux, tăng tốc độ xử lý và khả năng khai thác các tài nguyên của hệ thống tính toán. Đặc biệt, độ bảo mật của Solaris 10 được cung cấp ở mức cao hơn (mức-quốc-phòng) và tăng cường tính năng sẵn sàng. Đây là những tính năng rất quan trọng khi khách hàng ngày càng sử dụng mạng nhiều.

Các tính năng chính của Solaris 10:

1. DTrace - công cụ chẩn đoán lỗi mới trong Solaris 10. Người phát triển có thể loại bỏ các vấn đề về tốc độ xử lý và các lỗi khó phát hiện, chẩn đoán các vấn đề này "trong vài giây và vài phút".

2. Solaris Containers - cung cấp nhiều phân hoạch mềm với trên 8.000 bộ chứa khác nhau trên một biến thể của hệ điều hành. Tài nguyên có thể được tự động tái phân bổ và có thể đạt tới hiệu số sử dụng là 80%.

3. Process Rights Management - Solaris 10 đưa vào công nghệ bảo mật của hệ điều hành Trusted Solaris (TM) được sử dụng rộng rãi cho bảo mật việc truyền thông của chính phủ Mỹ. Process Rights Management sử dụng mô hình phân quyền chi tiết hơn giúp đảm bảo các lỗ hổng trong phần mềm không bị khai thác dẫn đến sự phá hủy hệ thống.

4. Predictive Self Healing - có thể tự động chẩn đoán, phân lập và phục hồi từ rất nhiều lỗi phần cứng và ứng dụng giúp giảm đáng kể thời gian ngừng hoạt động của hệ thống.

5. Linux Application Environment (với mã hiệu trước đây là Project Janus) - Solaris 10 là hệ điều hành duy nhất có thể thực hiện mã lệnh của Solaris và Linux đồng thời mà không hề phải sửa đổi, cung cấp cho khách hàng khả năng bảo vệ đầu tư và có nhiều ứng dụng hơn trên cả hai hệ điều hành này.

6. ZFS - với 16 tỷ tỷ lần lớn hơn so với hệ thống file 64-bit, ZFS sẽ là hệ thống file duy nhất không có giới hạn về dung lượng lưu trữ giúp Solaris trở thành hệ điều hành tốt cho lưu trữ dữ liệu. ZFS giúp giảm 80% công việc quản trị và cung cấp tính toàn vẹn dữ liệu tới khả năng phát hiện và sửa lỗi chính xác.

7. Cryptographic Framework - hệ thống mã hóa giúp tăng cường tốc độ xử lý và thực hiện quản lý tập trung việc mã hóa dữ liệu. Người phát triển ứng dụng có thể viết trên các API chung, giúp các khách hàng có thể bảo vệ đầu tư của mình và giúp các đối tác của Sun có thể viết các ứng dụng mà không cần lo ngại về việc triển khai hệ thống mã hóa.

8. Sun Update Connection - dịch vụ nâng cấp cho phép người sử dụng có thể phân tích hệ thống và luôn có được bản nâng cấp mới nhất từ bất cứ nơi nào. Sun Update Connection gửi các thông báo và cho phép người sử dụng có thể từ một máy của mình có thể nâng cấp tới nhiều máy ở các địa điểm khác nhau.

9. AMD64 - trên nền x86, Solaris 10 được tối ưu cho bộ xử lý AMD Opteron 64-bit, cung cấp khả năng xử lý cao hơn rất nhiều so với các hệ thống có chi phí thấp trước đây và đồng thời cung cấp khả năng tương thích nhị phân giữa môi trường 32-bit và 64-bit.

10. Đảm bảo tính tương thích: Sun cung cấp Solaris Application Guarantee để đảm bảo các ứng dụng viết cho các phiên bản trước của Solaris sẽ chạy trên Solaris 10.

Reduce Downtime with rsync

rsync is one of those tools that you find new uses for every day. It is really just a fancy file mover but it has an amazing feature set. Among other things You can control bandwidth usage, resume operations, copy files over ssh and include and exclude files as you like.

In The Practice of System and Network Administration they have a whole chapter dedicated to planning downtime (chapter 12 in the first edition). As a sysadmin it is your job to minimize it as much as possible but without sacrificing safety. In some cases rsync can be the perfect tool for this.

One usage scenario where I come back over and over again to rsync is when I want to move an application to a new machine, e.g a mail server. In a previous life I wanted to move a Cyrus imap server to a new hardware platform. At that time we had almost a 30GB mail spool and it would take an estimated 10 hours to move all files to the new hardware platform. The problem with Cyrus (and one of its strengths) is that it saves every email as a separate file. 10 hours was not really acceptable as a downtime window so an alternate solution was needed. In the end I used rsync with a total of 1 hour of downtime. Let me explain how.

I first shared the current mail spool using NFS to the new server. In this example the old mail spool is under /mnt and the new mail spool will be located under /cyrus. Late one evening I started my first rsync command.

# cd /cyrus
# rsync -arv /mnt .

This rsync will copy all files from /mnt to /cyrus and preserve all permissions and owner/group settings. This took about 10 hours to run and I did it while the mail server was active. So in the morning I had a copy of my mail spool but the copy was of course not consistent.

The next evening we did a trial run of the final migration. I ran this command and timed it carefully.

# cd /cyrus
# rsync -arv --delete /mnt .

Notice the –delete flag. It will delete all files in the destination that no longer exists in the source. This is to make sure that all emails the users deleted on the old server during the previous day are deleted on the new server as well. Since I timed this command I got a good approximation on how long the operation will last. It was around 1 hour. Now it was then time to announce the downtime window to the users. Fortunately I had very flexible users back then and 1 day warning for night work was ok.

The next evening about 2 hours before the downtime window I ran the previous command again. It sync:ed the mail spool once again over to the new server. When the downtime window started I disabled the mail server software on the old server (so I could get a final consistent copy) and then I ran the previous rsync command a third time and since it was just an hour since the previous once finished it only took 15 minutes. I then had to do some changes to internal DNS servers and bring up the Cyrus software on the new software and everything was ready.

So with a little bit of planning and careful use of rsync I could reduce my downtime window from 10 hours to 1 hour.

Tuesday, December 30, 2008

Managing Disk Space with LVM

The Linux Logical Volume Manager (LVM) is a mechanism for virtualizing disks. It can create "virtual" disk partitions out of one or more physical hard drives, allowing you to grow, shrink, or move those partitions from drive to drive as your needs change. It also allows you to create larger partitions than you could achieve with a single drive.

Traditional uses of LVM have included databases and company file servers, but even home users may want large partitions for music or video collections, or for storing online backups. LVM and RAID 1 can also be convenient ways to gain redundancy without sacrificing flexibility.

This article looks first at a basic file server, then explains some variations on that theme, including adding redundancy with RAID 1 and some things to consider when using LVM for desktop machines.

LVM Installation

An operational LVM system includes both a kernel filesystem component and userspace utilities. To turn on the kernel component, set up the kernel options as follows:

Device Drivers --> Multi-device support (RAID and LVM)

[*] Multiple devices driver support (RAID and LVM)
< > RAID support
<*> Device mapper support
< > Crypt target support (NEW)

You can usually install the LVM user tools through your Linux distro's packaging system. In Gentoo, the LVM user tools are part of the lvm2 package. Note that you may see tools for LVM-1 as well (perhaps named lvm-user). It doesn't hurt to have both installed, but make sure you have the LVM-2 tools.

LVM Basics

To use LVM, you must understand several elements. First are the regular physical hard drives attached to the computer. The disk space on these devices is chopped up into partitions. Finally, a filesystem is written directly to a partition. By comparison, in LVM, Volume Groups (VGs) are split up into logical volumes (LVs), where the filesystems ultimately reside (Figure 1).

Each VG is made up of a pool of Physical Volumes (PVs). You can extend (or reduce) the size of a Volume Group by adding or removing as many PVs as you wish, provided there are enough PVs remaining to store the contents of all the allocated LVs. As long as there is available space in the VG, you can also grow and shrink the size of your LVs at will (although most filesystems don't like to shrink).

Thumbnail, click for full-size image.
Figure 1. An example LVM layout (Click to view larger image)

Example: A Basic File Server

A simple, practical example of LVM use is a traditional file server, which provides centralized backup, storage space for media files, and shared file space for several family members' computers. Flexibility is a key requirement; who knows what storage challenges next year's technology will bring?

For example, suppose your requirements are:

400G  - Large media file storage
50G - Online backups of two laptops and three desktops (10G each)
10G - Shared files

Ultimately, these requirements may increase a great deal over the next year or two, but exactly how much and which partition will grow the most are still unknown.

Disk Hardware

Traditionally, a file server uses SCSI disks, but today SATA disks offer an attractive combination of speed and low cost. At the time of this writing, 250 GB SATA drives are commonly available for around $100; for a terabyte, the cost is around $400.

SATA drives are not named like ATA drives (hda, hdb), but like SCSI (sda, sdb). Once the system has booted with SATA support, it has four physical devices to work with:

/dev/sda  251.0 GB
/dev/sdb 251.0 GB
/dev/sdc 251.0 GB
/dev/sdd 251.0 GB

Next, partition these for use with LVM. You can do this with fdisk by specifying the "Linux LVM" partition type 8e. The finished product looks like this:

# fdisk -l /dev/sdd

Disk /dev/sdd: 251.0 GB, 251000193024 bytes
255 heads, 63 sectors/track, 30515 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Start End Blocks Id System
/dev/sdd1 1 30515 245111706 8e Linux LVM

Notice the partition type is 8e, or "Linux LVM."

Creating a Virtual Volume

Initialize each of the disks using the pvcreate command:

# pvcreate /dev/sda /dev/sdb /dev/sdc /dev/sdd

This sets up all the partitions on these drives for use under LVM, allowing creation of volume groups. To examine available PVs, use the pvdisplay command. This system will use a single-volume group named datavg:

# vgcreate datavg /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1

Use vgdisplay to see the newly created datavg VG with the four drives stitched together. Now create the logical volumes within them:

# lvcreate --name medialv  --size 400G
# lvcreate --name backuplv --size 50G
# lvcreate --name sharelv --size 10G

Without LVM, you might allocate all available disk space to the partitions you're creating, but with LVM, it is worthwhile to be conservative, allocating only half the available space to the current requirements. As a general rule, it's easier to grow a filesystem than to shrink it, so it's a good strategy to allocate exactly what you need today, and leave the remaining space unallocated until your needs become clearer. This method also gives you the option of creating new volumes when new needs arise (such as a separate encrypted file share for sensitive data). To examine these volumes, use the lvdisplay command.

Now you have several nicely named logical volumes at your disposal:

/dev/datavg/backuplv     (also /dev/mapper/datavg-backuplv)
/dev/datavg/medialv (also /dev/mapper/datavg-medialv)
/dev/datavg/sharelv (also /dev/mapper/datavg-sharelv)
Understanding the Linux Kernel

When a Volume Group Will Not Activate

Normally, volume groups are automatically activated during system startup. Unless you intentionally deactivate a volume group using vgchange, you will probably not need to reactivate a volume group.

However, LVM does require that a "quorum" of disks in a volume group be available. During normal system operation, LVM needs a quorum of more than half of the disks in a volume group for activation. If, during run time, a disk fails and causes quorum to be lost, LVM alerts you with a message to the console, but keeps the volume group active.

If there is no other way to make a quorum available, the -q option to the vgchange command will override the quorum check.

EXAMPLE:

vgchange -a y -q n /dev/vg01
You should attempt to return the disabled disks to the volume group as soon as possible. When you return a disk to service that was not online when you originally activated the volume group, use the activation command again to attach the now accessible disks to the volume group.

EXAMPLE:

vgchange -a y /dev/vg01

I had this problem of activating a volume group taking disks from network. so i did following steps

1)before reboot we need to disable volume groups, if one or more disks of volume group will not be present at boot time
vgchange -a n /dev/vg01
2) after booting system get the disk from across the system.
3) again activate the volume group.
vgchange -a y /dev/vg01

Understand ZFS in Solaris

Since the last post i have been working on zfs. Had even given a ppt on ->
1)Features of ZFS
2) ZFS Layers
3) VFS Interface
4) Basic Structures
5) Pool creation
6) Snapshot creation

5 & 6 next blog post

Features of ZFS
Immense capacity, 128 bit filesystem
Uses COW semantics
Pooled storage
Snapshots, clones, disk scrubs , resilvering, etc..

ZFS Layers




















JNI : Java Interface to libzfs

Interface Layer
ZPL (ZFS POSIX Layer): Filesystem abstraction
ZVOL : Volume Emulator

Transactional Object Layer
DMU (Data Management Unit): objects, objsets, transactions
DSL (Dataset & Snapshot Layer): managing snapshots & clones. reservations & quotas
ZAP (ZFS Attribute Processor): stores pool wide props, implements directory
ZIL (ZFS Intent Log): contains per dataset transaction logs

Pooled Storage Layer
ARC (Adaptive Replacement Cache): allows fs to share data with snapshots and clones
ZIO (ZFS I/O Pipeline): IO pipeline, Compression and checksums can be done
VDEV (Virtual Devices): unified access to devices
SPA (Storage Pool Allocator): managing pools

LDI (Layered Driver Interface) : interacts with underlying physical devices

VFS Interface














Basic Structures
Uberblock : Similar to superblock.











Dnode : Objects are defined in this structure.








Znode : Stores the attributes for the filesystem object.

uint64_t zp_atime[2];
uint64_t zp_mtime[2];
uint64_t zp_ctime[2];
uint64_t zp_crtime[2];
uint64_t zp_gen;
uint64_t zp_mode;
uint64_t zp_size;
uint64_t zp_parent;
uint64_t zp_links;
uint64_t zp_xattr;
uint64_t zp_rdev;
uint64_t zp_flags;
uint64_t zp_uid;
uint64_t zp_gid;

uint64_t zp_pad[4];
zfs_znode_acl_t zp_acl;

MOS – (Meta Object Set) root of all pool-wide metadata.


Moving a volume group to another system

It is quite easy to move a whole volume group to another system if, for example, a user department acquires a new server. To do this we use the vgexport and vgimport commands.

vgexport/vgimport is not necessary to move drives from one system to another. It is an administrative policy tool to prevent access to volumes in the time it takes to move them.
Unmount the file system
First, make sure that no users are accessing files on the active volume, then unmount it

# unmount /mnt/design/users

Mark the volume group inactive
Marking the volume group inactive removes it from the kernel and prevents any further activity on it.

# vgchange -an design
vgchange -- volume group "design" successfully deactivate

Export the volume group

It is now necessary to export the volume group. This prevents it from being accessed on the ``old'' host system and prepares it to be removed.

# vgexport design
vgexport -- volume group "design" successfully exported

When the machine is next shut down, the disk can be unplugged and then connected to it's new machine

Import the volume group

When plugged into the new system it becomes /dev/sdb so an initial pvscan shows:

# pvscan
pvscan -- reading all physical volumes (this may take a while...)
pvscan -- inactive PV "/dev/sdb1" is in EXPORTED VG "design" [996 MB / 996 MB free]
pvscan -- inactive PV "/dev/sdb2" is in EXPORTED VG "design" [996 MB / 244 MB free]
pvscan -- total: 2 [1.95 GB] / in use: 2 [1.95 GB] / in no VG: 0 [0]

We can now import the volume group (which also activates it) and mount the file system.
If you are importing on an LVM 2 system, run:

# vgimport design
Volume group "vg" successfully imported

If you are importing on an LVM 1 system, add the PVs that need to be imported

# vgimport design /dev/sdb1 /dev/sdb2
vgimport -- doing automatic backup of volume group "design"
vgimport -- volume group "design" successfully imported and activated

Activate the volume group

You must activate the volume group before you can access it.

# vgchange -ay design

Mount the file system

# mkdir -p /mnt/design/users
# mount /dev/design/users /mnt/design/users

The file system is now available for use.

Crontab jobs on solaris 10

Here is some ways to enabled cron job on solaris 10.
bash-3.00# export EDITOR=vi
bash-3.00# crontab -e root

* * * * * command to be executed

- - - - -
| | | | |
| | | | +—– day of week (0 - 6) (Sunday=0)
| | | +——- month (1 - 12)
| | +——— day of month (1 - 31)
| +———– hour (0 - 23)
+————- min (0 - 59)

Example:
* * * * * /apps/logs/bluecoat/bcscript.sh
configuration above will execute “bcscript.sh” script on “/apps/logs/bluecoat/” every minute
restart the cron service daemon with the following command:
bash-3.00# svcadm restart cron
bash-3.00# svcs cron

Read ext3 USB disk in Solaris

I often get queries about mounting NTFS / Ext2 / Ext3 / FAT 16 / FAT 32 partions in Solaris . Follow the below steps . Note that its only read-only support for NTFS / Ext2 / Ext3 .

* Download packages
FSWpart
FSWfsmisc

Note these are packages for x86 Solaris .

* Unzip the packages and install it

root~#gzcat FSWpart.tar.gz | tar xvf -

root~#gzcat FSWfsmisc.tar.gz | tar xvf -

root~#pkgadd -d . FSWpart

root~#pkgadd -d . FSWfsmisc

* Now run the prtpart tool

root~#/usr/sbin/prtpart

It would result in following output
root~#/usr/sbin/prtpart

Fdisk information for device /dev/rdsk/c1d0p0

Block Size : 512 bytes
Controller : ide
Disk : cmdk
Capacity : 74 GB

# start block # nblocks startCylSecHd endCylSecHd OSType
1: 0000000063 0006152832 0/ 1/ 1 7e/7f/fe Diagnostic
2: 0006152895 0025173855 7f/41/ 0 ff/ff/fe WIN95 FAT32(LBA)
3: 0031328640 0052432128 ff/ff/fe ff/ff/fe Solaris x86
4: 0083760830 0072535555 ff/ff/fe ff/ff/fe DOS Extended

# start block # nblocks startCylSecHd endCylSecHd OSType
5: 0000000001 0031407012 ff/ff/fe ff/ff/fe Linux native
0: 0031407013 0001994202 ff/ff/fe ff/ff/fe DOS Extended

# start block # nblocks startCylSecHd endCylSecHd OSType
6: 0000000063 0001994139 ff/ff/fe ff/ff/fe Linux swap
0: 0033401215 0039134340 ff/ff/fe ff/ff/fe DOS Extended

# start block # nblocks startCylSecHd endCylSecHd OSType
7: 0000000063 0039134277 ff/ff/fe ff/ff/fe WIN95 FAT32(Upto 2047GB)


Now note the disk id. In the above example its
"/dev/rdsk/c1d0p0"

Now run

root~#/usr/sbin/prtpart /dev/rdsk/c1d0p0 -ldevs

Fdisk information for device /dev/rdsk/c1d0p0

** NOTE **
/dev/dsk/c1d0p0 - Physical device referring to entire physical disk
/dev/dsk/c1d0p1 - p4 - Physical devices referring to the 4 primary partitions
/dev/dsk/c1d0p5 ... - Virtual devices referring to logical partitions

Virtual device names can be used to access EXT2 and NTFS on logical partitions

/dev/dsk/c1d0p1 Diagnostic
/dev/dsk/c1d0p2 WIN95 FAT32(LBA)
/dev/dsk/c1d0p3 Solaris x86
/dev/dsk/c1d0p4 DOS Extended
/dev/dsk/c1d0p5 Linux native
/dev/dsk/c1d0p6 Linux swap
/dev/dsk/c1d0p7 WIN95 FAT32(Upto 2047GB)

The above command scan you hard-disk and displays all available filesystems. Note the required filesystems to be mounted.

To mount NTFS partition use

root~#mount -F ntfs /dev/dsk/c1d0p /mnt/windows

To mount FAT 16 / FAT 32 partition use

root~#mount -F pcfs /dev/dsk/c1d0p /mnt/windows

if the above command fails you can try the below option

root~#prtpart /dev/dsk/c1d0p0 -fat

the above command should list the available PCFS / FAT partitions in colon notation, then use the same for mounting (eg)

root~#mount -F pcfs /dev/dsk/c1d0p0:d /mnt/windows

To mount Ext2 / Ext3 partitions use

root~#mount -F ext2fs /dev/dsk/c1d0p /mnt/linux

To unmount a partition use "umount "

root~#umount /mnt/linux

LinkWithin

Related Posts with Thumbnails