Shrink partition in xen guest

April 25th, 2015

In a previous post I discussed extending a partition in a linux guest. Since I never can make up my mind, this post discusses the opposite, i.e. shrinking a paritition in a disk (again as opposed to a partition) in a logical volume. Since we do not have just one partition in the logical volume we have to delete and recreate.

The procedure is a little different since it is too large to make a backup in the other quest I do it on the xen host:

  • Create a logical volume the size of the one we are going to rearrange for backup
  • Mount the disk in the Logical Volume (LV) on another guest.
  • In the other guest use fdisk to recreate them with the right size

 

lvcreate -L 100G -n lv_vm_tmp /dev//

For instance:

lvcreate -L 100G -n lv_vm_ishpublic_tmp /dev/nebulus-vg

Backup the original volume to the temp volume we just created (double check that the original is first and the backup is second).

cp /dev// /dev//
cp /dev/nebulus-vg/lv_vm_ishpublic /dev/nebulus-vg/lv_vm_ishpublic_tmp

Now add this to your ‘utility’ host (we are going to mount the volume there to resize). In your xen config for utility add the disk. For instance my volume group is ‘nebulus-vg’ and the disk with partitions I want to resize is ‘lv_vm_ishpublic’.

disk = [ '/dev/nebulus-vg/lv_sirius_ubuntu,raw,xvda,rw', '/dev/nebulus-vg/lv_vm_ishpublic,raw,xvdb,rw' ]

Then:

  • Start your utility VM
  • login as root

Print the partitions on xvdb

root@sirius:~# fdisk /dev/xvdb

Welcome to fdisk (util-linux 2.25.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/xvdb: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x76ff0726

Device     Boot     Start       End   Sectors Size Id Type
/dev/xvdb1           2048 201328639 201326592  96G 83 Linux
/dev/xvdb2      201330686 209713151   8382466   4G  5 Extended
/dev/xvdb5      201330688 209713151   8382464   4G 82 Linux swap / Solaris

Partition 3 does not start on physical sector boundary.


Command (m for help): q

Run a file system check:

e2fsck -f /dev/xvdb1

Try resizing the partition (we can pass a very low number to resize2fs so it will print the minimum we need:

resize2fs /dev/xvdb1 1G
resize2fs 1.42.10 (18-May-2014)
resize2fs: New size smaller than minimum (4468729)

You probably do not want to have the exact minimum, I use approx. double the size.

resize2fs /dev/xvdb1 8388608
resize2fs 1.42.10 (18-May-2014)
Resizing the filesystem on /dev/xvdb1 to 8388608 (4k) blocks.
The filesystem on /dev/xvdb1 is now 8388608 blocks long.

Though the filesystem is resized the partition is still the original size. We need to modify the partition table on the disk where the new end is. Calculate this from the output of resize2fs and the size of a sector from fdisk (sector size was 512 in my case, make sure you substitute the values you got!).

  • New filesystem size (output from resize2fs) = 8388608 (4k) blocks = 8388608 * 4 * 1024 = 34359738368 bytes
  • New filesystem size in sectors = 34359738368 / 512 = 67108864 sectors (if you get a fractional value round it up)
  • Therefore the end sector in fdisk will be start sector + size = 2048 + 67108864 = 67110912
  • I also want a 2G swap partition i.e. 2 * 1024 * 1024 * 1024 / 512 = 4194304 sectors
  • Therefore the end of the disk will be at 67110912 + 4096 = 71305217 sectors
 fdisk /dev/xvdb

Welcome to fdisk (util-linux 2.25.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/xvdb: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x76ff0726

Device     Boot     Start       End   Sectors Size Id Type
/dev/xvdb1           2048 201328639 201326592  96G 83 Linux
/dev/xvdb2      201330686 209713151   8382466   4G  5 Extended
/dev/xvdb5      201330688 209713151   8382464   4G 82 Linux swap / Solaris

Partition 3 does not start on physical sector boundary.


Command (m for help): d
Partition number (1,2,5, default 5):

Partition 5 has been deleted.

Command (m for help): d
Partition number (1,2, default 2):

Partition 2 has been deleted.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Create the new partitions:


 

Command (m for help): n Partition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions) Select (default p): Using default response p. Partition number (1-4, default 1): First sector (2048-209715199, default 2048): Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199): 67110912 Created a new partition 1 of type 'Linux' and of size 32 GiB. Command (m for help): n Partition type p primary (1 primary, 0 extended, 3 free) e extended (container for logical partitions) Select (default p): Using default response p. Partition number (2-4, default 2): First sector (67110913-209715199, default 67112960): 67110913 Last sector, +sectors or +size{K,M,G,T,P} (67110913-209715199, default 209715199): 71305217 Created a new partition 2 of type 'Linux' and of size 2 GiB.

Toggle the filesystem type on the swap partition:

t 2 82

Write the table back to disk, quit and shutdown the utility system:


 

w q shutdown -h now

Back on the xen host we now need to reduce the logical volume size. First we need to know the physical extent:

 vgdisplay
  --- Volume group ---
  VG Name               nebulus-vg
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  12
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                7
  Open LV               5
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               3.64 TiB
  PE Size               4.00 MiB
  Total PE              953672
  Alloc PE / Size       129825 / 507.13 GiB
  Free  PE / Size       823847 / 3.14 TiB
  VG UUID               tm4Q7z-LWIL-OUW3-egF0-8UP4-d2MJ-fW1sXX


PE Size is 4 MB

To calculate the new size in extends take the size in bytes from our last sector ( I am not sure if sectors start from 1 or 0, better be safe) from fdisk and add one.

  • 71305217 + 1 = 71305218

 

Round up to the next vg extent boundary:

  • RoundUp( Number of sectors * sectors size / PE Size )
  • 71305218 * 512 / ( 4 * 1024 * 1024 ) = 8704.25
  • RoundUp( 8704.25 ) = 8705

Shrink the logical volume

lvreduce -l 8705 /dev/nebulus-vg/lv_vm_ishpublic
  WARNING: Reducing active logical volume to 34.00 GiB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce lv_vm_ishpublic? [y/n]: y
  Reducing logical volume lv_vm_ishpublic to 34.00 GiB
  Logical volume lv_vm_ishpublic successfully resized

And we are done. Well actually you would want to startup your utility VM again, do the mount of the filesystem and run e2fsck (see above) again to make sure that everything worked.

e2fsck /dev/xvdb1
e2fsck 1.42.10 (18-May-2014)
/dev/xvdb1: recovering journal
Setting free inodes count to 1775009 (was 1775029)
Setting free blocks count to 4183513 (was 4183757)
/dev/xvdb1: clean, 322143/2097152 files, 4205095/8388608 blocks

Bonus points enable swap.

Shutdown util and startup your VM with resized disk and login:

If you configured a swap parition check it is working:

swapon -s

If not you have to add the swap partition. Let’s say it was on xvdb2 when you configured it in util then it probably is in xvda2 (make sure you get it right i.e. check if it exists in /dev isn’t mounted df -k). In the following I assume it is on /dev/xvda2

Now for ubuntu execute the following:

blkid /dev/xvda2

Copy the UUID and open /etc/fstab. Copy the swap line and change it as follows (use the printed UUID instead of the one below).

UUID=7340ea5f-3325-466a-8e2f-52c1cf15ae94 / ext4 errors=remount-ro 0 1
# swap was on /dev/xvda5 during installation
#UUID=75ecaea3-ff0d-4f90-b5e2-ea757bde5399 none swap sw 0 0
UUID=582dae7a-5d2a-4b4c-9685-993fc8454200 /dev/xvda2 swap sw 0 0

shutdown -r now

Log back in:

swapon -s
Filename                                Type            Size    Used    Priority
/dev/xvda2                              partition       1046524 0       -1

If satisfied don’t forget to start Util and cleanup the backups created.

Extend partition in disk in xen linux guest

April 12th, 2015

After searching the web for a method to resize a partition in a disk in a logical volume as opposed to just a partition in a logical volume (notice the difference ‘partition in’ or ‘disk in’). I could not find a method however if you have enough space you could use the following method.

Note that I used this to resize the partition of a virtual machine running in a logical volume (it should work with physical stuff too but then you might have to put a disk in another physical host or mount it using a live cd).

Util is a machine with the same version of the OS. tbe is the one we want to extend.

From a high level the procedure is as follows:

  • Mount the disk with partitions you want to extend on another virtual machine (VM)
  • Use cp to backup the disks
  • Use fdisk to extend the partitions
  • Use cp to restore the backup to the resized partitions
  • On the VM with, now, resized disks, run resize2fs
  • Bonus; resize and or add the swap partition

Everything is done as root. Prefix with sudo if you want or need to.

xl shutdown util 
xl shutdown tbe 

Extend the disk on the host.

vgs
lvs
lvextend -L8G /dev/<vg>/<lvm>

Unfortunately although the logical volume is extended the disk inside isn’t using the extra size. Since we do not simply mount the disk but have a complete disk within the lv we cannot simply use resize something. Instead we have to mount it on, preferably the same version of the OS and then make a backup of the partition(s). Drop everyting and create new partitions then copy it back:

edit util .cfg

vgs
lvs
disk = [ '/dev/<VG>/<lv util>,raw,xvda,rw', '/dev/<VG>/<lv tbe>,raw,xvdb,rw' ]

Bring our util system backup

xl create util.cfg

Login to util then:

ls -tarl /dev/xvdb*

Make a backup of all the regular partitions (no need to backup swap of course).

mkdir /backup
# Repeat for all other partitions (you could skip swap if you want but I would be on the safe side and backup all just in case you got the partition wrong)
cp /dev/xvdb1 /backup/xvdb1backup1
cp /dev/xvdb2 /backup/xvdb1backup2
# etc.. 
fdisk /dev/xvdb

Command (m for help): p
Disk /dev/xvdb: 14 GiB, 15032385536 bytes, 29360128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x20230da1

Device Boot Start End Sectors Size Id Type
/dev/xvdb1 2048 8388607 8386560 4G 83 Linux
/dev/xvdb2 8388608 16777215 8388608 4G 82 Linux swap / Solaris


Now remove all the partitions listed (you didn’t skip the backup step, right…).

Command (m for help): d
Partition number (1,2, default 2):

Partition 2 has been deleted.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

 

Then create new partitions make sure specify the size you want. The calculation is as follows:

Let’s say we want two partitions.

  1. Primary partition 13GB
  2. Swap partition remainder of the disk.

For the start value we use the start value that fdisk will print when pressing n, p. The number we have to add to this to get a 13GB partition is as follows. The first partition must be a primary partition. The following can be extended (if you need more than 3 other partitions) or more primary ones. The sector size is printed by fdisk (see above):

Diff = PartitionSizeInGB * 1024 * 1024 *1024 / SectorSizeInBytes

For instance

Diff = 13 * 1024 * 1024 * 1024 / 512 = 27262976

Therefore the last sector is:

LastSector = 27262976 + 2048 = 27265024

 

Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-29360127, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-29360127, default 29360127): 27265024

Created a new partition 1 of type 'Linux' and of size 13 GiB.

And lastly create a swap partition using the rest of the disk i.e. just go with the default values.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2):
First sector (27265025-29360127, default 27267072):
Last sector, +sectors or +size{K,M,G,T,P} (27267072-29360127, default 29360127):

Created a new partition 2 of type 'Linux' and of size 1022 MiB.

We now have to change the type to linux swap which has code 82.

Command (m for help): t
Partition number (1,2, default 2):
Hex code (type L to list all codes): 82

Changed type of partition 'Linux' to 'Linux swap / Solaris'.

Command (m for help):

Write the results to disk:

w

Copy the backup back to the disk

cp /backup/xvdb1backup1 /xvdb1
#etc for all partitions

Shutdown util

shutdown -h now

In the host remove the disk again from util.cfg Now start the tbe

xl create tbe.cfg

Logon to tbe

df -k 

Still no luck huh? 😉

su - root
resize2fs /dev/<the device you extended>
df -k

Should now list the new size.

If you configured a swap parition check it is working:

swapon -s

If not you have to add the swap partition. Let’s say it was on xvdb2 when you configured it in util then it probably is in xvda2 (make sure you get it right i.e. check if it exists in /dev isn’t mounted df -k). In the following I assume it is on /dev/xvda2

Now for ubuntu execute the following:

blkid /dev/xvda2

Copy the UUID and open /etc/fstab. Copy the swap line and change it as follows (use the printed UUID instead of the one below).

UUID=7340ea5f-3325-466a-8e2f-52c1cf15ae94 / ext4 errors=remount-ro 0 1
# swap was on /dev/xvda5 during installation
#UUID=75ecaea3-ff0d-4f90-b5e2-ea757bde5399 none swap sw 0 0
UUID=582dae7a-5d2a-4b4c-9685-993fc8454200 /dev/xvda2 swap sw 0 0

shutdown -r now

Log back in:

swapon -s
Filename                                Type            Size    Used    Priority
/dev/xvda2                              partition       1046524 0       -1

If satisfied don’t forget to start Util and cleanup the backups created.

Converting Eclipse Android Project with Libraries and VCS to AndroidStudio

February 20th, 2015

This thing puzzled me for a while. I have several separate libraries in eclipse and their associated SVN directories. I

  1. Did not want to duplicate sources in the repo
  2. Wanted everything under source control
  3. Wanted to use Gradle (because that seems to be Google’s preferred source build system for android)
  4. Did not mind having everything in one project instead of separate modules in Android Studio ((un)like I had in eclipse)

So what I did was the following:

  1. Import project from Eclipse by choosing ‘Import from eclipse’ from the main AndroidStudio window.
  2. Go into the project and for all sub-projects in AndroidStudio open the Java folder and delete the Java source tree (i.e. the Java directory should be empty)
  3. Warning for the next step; in your repository, make sure to create a directory for your project. Do not import into the root or it will be a mess!
  4. Check in your (now empty) android studio project by going to (Toolbar) VCS->Subversion->Share Project. So your, Android project, will be under source control in its own folder.
  5. Now for every subproject click on (Toolbar) VCS->Checkout->Browse to the root of your source (i.e. the last part of your domain name, for me it is ‘eu’)
  6. Browse to your project, subproject and then the java folder.
  7. Press Checkout
  8. Choose the option with the last directory being Java followed by the first part of your domain name (i.e. in my case again ‘eu’). So for me …Java/eu
  9. Then select ‘NO’, do not create an android studio project from it.

You should be all set. When you now commit the project itself it will go into its VCS location while the subprojects will update the existing VCS project locations.

Bonus points for anyone who figures out to do this, but then with a separate android studio project for every library. 🙂

 

 

The Imaginarium of Doctor Parnassus

November 25th, 2011

Technical details:

* Genre: Fantasy (the back of the casing says adventure)

* Play time: 118 minutes

* Sound English DTS 5.1, English Dolby 5.1

* Subtitles: Dutch

This wonderful tale brings you to worlds imagined by Doctor Parnassus. Due to a pact with the devil, he is immortal. Obviously there is a price to pay for that, but I won’t go into details because I think this is not what you buy this movie for. For me the value is in the lovely scenery, the beautifully imagined worlds and the original story. A very unique movie with an exceptional tale. If you value fantasy a little bit more than a good story then you enjoy 118 minutes of excitement. Anyone who is interested in something different than the usual movie willl find value in this picture.

 

I can recommend this movie to any fan of the fantasy genre.

Robots

November 25th, 2011

This is a 3D animated movie. The story is about a robot moving to the big city and particularly ‘Big Wigs’  uh, I mean ‘Big Welders’ factory. However when arriving there, things aren’t as he expected them to be. This movie has an original story and some very nice graphics. I especially liked how they textured the ‘suit’ robots (stainless steel) and the ordinary robots which looked much like the tin equipment you had back in the 60’s and 70’s with frayed paint and round chrome edges.

A fun 3D animated movie with a lot of humor and an original story, that I can recommend to any fan of 3D animated movies.

The Legend of Zorro

September 20th, 2011

Technical details:

  • Voices: English
  • Subtitles: English, Dutch, Hindi
  • Dolby 5.1
  • Extras: Deleted scenes, Commentaries, Making of etc.
Legend of Zorro

Legend of Zorro

Back in the 70-ties, when I watched zorro on my parent’s black and white TV set, about 7 years old, and discovering the world around me, two questions taunted me.

  • What color was Zorro’s cape,
  • and why for heaven’s sake was his image on a bottle of sherry?

It took quite some years to find out that his cape was just black (what a disappointment). Why his image was associated with sherry, was something this movie might reveal.

You know being a superhero is just not as easy as you would think. Obviously there are problems with the kids, being away all the time due to your superhero duties, and the pressure that goes with it. Of course this imposes a lot of stress and one good way to fight stress is with alcohol. Now about 40 years later I finally understand why Zorro is on that bottle of sherry! Yup, he is a good candidate for AA.

Problems with the wife and kid, appearing drunk at a party. That pretty sums up this movie. Of course the bright candle lights, the barrels of wine, and warm nights makes for some good looking camera shots, however, it can’t save the movie. There is some superhero action at the beginning and end of the movie but nothing in between. I think they could have made a nice movie with the same effort. Not sure to whom this movie is supposed to appeal. People interested in social interaction aren’t likely to buy a movie with Zorro in the title, likewise someone who want to see a superhero solving the worlds problems isn’t going to be satisfied seeing his/(her) superhero succumb to alcohol.

If you are interested in a superhero’s social problems, then this, is the movie to watch, if not then just skip this one.

This DVD I got from the bargain bin, and that is where it should have stayed.

Final verdict, don’t buy.

 

The League of Extraordinary Gentlemen

September 20th, 2011

Technical details:

  • Voices: English, Turkish
  • Subtitles: English, Dutch, Danish, Fin, Icelandic, Nordic, Swedish, Turkish
  • PAL
  • 5.1 DTS
  • Playtime: 106 minutes
  • Extras: Comentaries, Removed scenes, Making of etc.
  • Type: Fantasy
  • Two disc edition
  • Price tag: between 7 and 10 €
League of Extraordinary Gentlemen

League of Extraordinary Gentlemen

I didn’t think too much of the  movie while reading the back of the box. “A captain, a hunter, a strong man, a vampire, an invisible man and someone who can’t die have to save the world”. Well it can’t get anymore ‘cliche’ than that. However this movie, based on a book, has all the right ingredients, humor, adventure and romance (ok your mileage may vary on that last one 🙂 ).

It is fun to see who the main characters really are, so I won’t spoil the fun here. This movie has a lot of FX and is really fun to watch. Although completely different, I can imagine, this would appeal to people who like a movie like ‘Indiana Jones’.

One of the main characters, played by Sean Connery, dies at the end. However it leaves the possibility of a sequel open, and I can only hope one will be created.

King Kong

September 19th, 2011

Technical details:

Voices: English

Subtitles: English, Dutch, Finish, Hebrew, Nordic, hungarian, Swedish

Playtime: 180 Minutes (Yup, that is not a typo 🙂 )

Price tag: 7 €

King Kong

When I see the title ‘King Kong’ I can’t happen to view a very clumsy ape attached to the Empire State Building. A scene typical of the ancient ‘King Kong’ movie. However this ‘King Kong’ movie is different. Set in the twentieths of the previous century, a group of people, brought together through circumstances, are heading to ‘Skull Island’. They’ll bring back King Kong to New York where he is on display. Of course in the end he will escape his prisson and the classical scene on the empire state building will play out.

This movie was directed by Peter Jackson, who also directed ‘Lord of The Rings’ and it shows. King Kong is really worth every penny you paid.

I really enjoyed this movie.

Star Trek 2009

September 19th, 2011

Star Trek 2009 has the following technical features:

  • Voices: English
  • Subtitles: Dutch, English, French
  • Playtime: 122 Minutes
  • Type: Scifi
  • Price tag: I believe 8 Euros or so.

Being a fan of the original star trek series, the ones with William Shatner as captain Kirk, I was a bit hesitant to buy a movie with a completely new cast. But reading some reviews I thought I would give it a chance. The story is in any case a typical ‘star trek’ story. Including Romulans, time travel and firing phasers. It has the fantasy of the original series but, and this contrary to the previous ‘Star Trek’ movies, the pace of a modern movie. To top if off the original Spock also plays a role as the ‘old’ spock. The film directed by JJ Abrams has a very distinguished look. Everything looks very dynamic, which is especially surprising when you see how they pulled it off. For the latter see the extras on the DVD.

I can recommend this movie to anyone who liked the original series.

 

Tin man review

August 29th, 2009

Tin man is a new take on the Wizard of Oz. The technical details are:

  • Voices: English
  • Subtitles: Dutch
  • Dolby: 5.1
  • Playtime: 180 minutes
  • Type: Fantasy
  • Price tag: 12 Euro

 

Tin Man DVD

Tin Man DVD

The story is about DG who has some strange dreams for some time. At a certain moment she goes to the ‘other side’. Once on the there she meets some odd people. Someone whose brain was removed a kind of lion and someone who got punished by being locked up in a tin frame. Together they travel to fight the evil Azkedellia.  On the ‘other side’ you will see shape shifting people, robots, magic etc. Together it will keep it interesting for the full 180 minutes.

Although it refers to the Wizard of Oz it actually is a complete different movie than any Wizard of Oz I have seen. And that is actually for the better. I thought the acting was quite good and the animations and special effects very good. You can of course watch the movie in one go. I watched it in two parts though. It had a good pace and never got boring or anything.  If you like special effects and a fantasy setting then this is certainly movie to buy, just forget that it has anything to do with the Wizard of Oz.