## Archive for the ‘Technobabble’ Category

Sunday, November 12th, 2017

For my DSL ‘s the preferred tool is ANLTR by Terence Parr and his team (although I sometimes like to hone my Flex/Lex, Bison/Yacc skills too). With the new ANLTR4 creating your own compiler front end has never been so easy.

It still took me a little bit of time to combine it with Eclipse and Gradle. I did the following to make it work:

• Install Antlr4 by Edgar Espina from eclipse market place
• Create ANLTR project New->General->ANLTR 4 Project
• Add grammar file e.g. src/main/antlr/<package>/mygrammar.g4
• Important. Add header to your grammar with package . This will make sure that the package structure is created in the jar file like in the following example:

grammar TimeParser;

package eu.informationsuperhighway.timeparser;
import java.util.HashMap;
}

• Right Click the project->Build Path->Source Path->Click ‘Add Source Path’->Click ‘Browse’
• Browse to build->generated-src->antlr->main and make sure the checkbox for main is checked. This is where the generated sources end up and if it is not listed as the source dir then the calling project won’t be able to find the sources! You might need to compile first before this directory shows up.
• You can right click on the <grammar>.g4 file to set options (although I had to restart eclipse to make them take effect). I you do a Gradle build the following Gradle file should do. You might need to upgrade Gradle so it recognizes the antlr4 plugin (execute task wrapper).
• I used the following Gradle file to create a library to include into other projects. I am only interested in the ‘Visitor Design Pattern’ so I added ‘-no-listener’ and ‘-visitor’  for the arguments parameter ( see end of this post).
• Note that if this is a library (probably), in the calling project, under dependencies add ‘compile project( ‘:<your project name>’ )’  and in settings.gradle add:

include ':<your project name>'


e.g.

include ':timeparser'
project(':timeparser').projectDir = new File(settingsDir, '../timeparser')

• Also in the calling project’s buildpath add this project as dependent project (so the jar gets included) Right Click project->Build Path->Configure Build Path->Projects-> Add the library project
• For verification; Note that the library’s project’s jar should be in workspace\<project>\build\libs.
/*
*
* This generated file contains a commented-out sample Java project to get you started.
* For more details take a look at the Java Quickstart chapter in the Gradle
* user guide available at https://docs.gradle.org/3.5/userguide/tutorial_java_projects.html
*/

// Apply the java plugin to add support for Java
apply plugin: 'antlr'
apply plugin: 'java-library'
// In this section you declare where to find the dependencies of your project
repositories
{
// Use 'jcenter' for resolving your dependencies.
// You can declare any Maven/Ivy/file repository here.
jcenter()
mavenCentral()
}

}

generateGrammarSource
{
maxHeapSize = "64m"
arguments += ["-visitor", "-long-messages", "-no-listener"]
}

// In this section you declare the dependencies for your production and test code
dependencies
{
// The production code uses the SLF4J logging API at compile time
//compile 'org.slf4j:slf4j-api:1.7.22'

// Declare the dependency for your favourite test framework you want to use in your tests.
// TestNG is also supported by the Gradle Test task. Just change the
// testCompile dependency to testCompile 'org.testng:testng:6.8.1' and add
// 'test.useTestNG()' to your build script.
testCompile 'junit:junit:4.12'

// https://mvnrepository.com/artifact/org.antlr/antlr4-runtime
compile group: 'org.antlr', name: 'antlr4-runtime', version: '4.7'

antlr "org.antlr:antlr4:4.7" // use ANTLR version 4
}

jar
{
archiveName = "timeparser.jar"
from
{

configurations.runtime.collect { it.isDirectory() ? it : zipTree(it) }

configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
}

from( "timeparser/build/generated-src/antlr/main" )

manifest
{
//attributes 'Main-Class': 'eu.informationsuperhighway.something.Main'
}

exclude 'META-INF/*.RSA', 'META-INF/*.SF','META-INF/*.DSA'
}


### Adding the Android Licensing Library to an AndroidStudio Project

Friday, May 13th, 2016

It appears that currently creating an external library is not all that easy. I wanted to add the Android Licensing Library and they advise creating a copy from it. I decided to just use a project library instead of messing around but have the sources in the repo. Here is what I did:

• Copy it from the android SDK (the location is visible in AndroidStudio->Tools->Android->SDK Manager) to another location
• Check in the library into your version control system
• In your AndroidStudio project create a library project. Projects->New Module->Library ‘SomeName’ Package name = com.google.android.vendor.licensing
• Now with your file system browser go into the ‘SomeName’->src->main->java directory and delete the package that was created
• Check out the licensing library into a new empty directory somewhere and go into the src dir and copy the ‘com’ subfolder
• Copy this tree to the  ‘SomeName’->src->main->java directory (this makes sure the .svn references are now pointing to the correct repo location)
dependencies
{
compile project(':assetLib')
compile project(':openGLISH')
compile project(':mathLib')
compile project(':ishlicensing')
compile 'com.android.support:support-v4:18.0.0'
}
• Now you should be able to compile your project and check everything into svn. (note uncheck ‘Code Analysis’ if pressing the ‘Commit’ button doesn’t do anything).

### Measuring Distance with an HC-SR04

Tuesday, February 16th, 2016

The HC-SR04 is a cheap (in terms of money)  module that can measure distances using ultrasound. The module uses a 40 KHz tone, which is of course way above what any human can hear, to measure the distance. It works as follows.

• It expects a 10µs TTL, i.e. 0-5V, pulse on its ‘Trigger’ input
• It will then send out a sound
• Waits until the sound comes back and calculate the distance
• Create a pulse on the ‘Echo’ output. The length of the pulse will correspond to the distance measured

I used a ATMega328p MCU to control the HC-SR04.

The yellow channel shows the outgoing pulse to the ‘Trigger’ input of the HCSR04. The blue channel shows the signal coming back on the ‘Echo’ output the length of it being proportional to the distance measure. The OLED display shows the values measured. The first few values are for debugging the last shows the measured and calculated distance in meters. In the top right the HCSR04 is visible sitting on my laser distance meter. Compare the value of that 0.156 to the one on the OLED display 0.150. They are pretty close.

The main parts of the program consist of:

• an ISR (Interrupt Service Routine) to be called when the echo pin goes to high
• a timer to measure the length of the pulse sent by the HC-SR04

One thing to keep in mind that it is not the time between starting the HC-SR04 and the start of the echo pulse that should be measured but the length of the echo pulse. That is to say it is not really the echo the HC-SR04 just sends you back the measure distance and the simplest way is to vary the length of the signal.

My ATMega328p is clocked at 8MHz. The supply voltate is 3.3V to stay compatible with my RPi. The HCSR04 however needs to be connected to a 5V supply. Obviously you need to connect the grounds of both power supplies to make it work. The ‘Trigger’ input was connected directly to the ATMega328p since it produces a 3.3V high and that is well within the limits of what is considered ‘high’ in TTL. However the ‘Echo’ pin I connected through a two resistor voltage divider (i.e. two resistors in series, one connected to the ‘Echo’ ouput and the other to ground while the middle is the ‘Echo’ that goes to the ATMega. The one connected to the ground divided by both, times the 5V should be somewhat below 3.3V.

Note that the code should be optimized, obviously, for any real application. Note also that the following is not complete (since I already deleted the code but I’ll recreate it some time and update this) just showing the idea. I used the OLED from a previous post for the output.

volatile uint16_t MeasureCount;
ISR (INT0_vect)
{
// Reset timer
TCNT1H = 0;
TCNT1L = 0;
// Wait until pind goes back to zero

while( PIND & EchoPin );

uint16_t LSB = TCNT1L;
uint16_t MSB = TCNT1H;
uint16_t Pulses = ( MSB << 8 ) | LSB;
//Pulses = 65535;
float Distance = ( (float) Pulses * 343.2f ) / ( 2.0f * 8000000.0f ) + 0.0005f;
uint16_t Whole = (int)Distance;
uint16_t Partial = (int)( ( Distance  - Whole ) * 1000.0f );
++MeasureCount;
sprintf( Message0, "M %06u H = %u L = %u %u", MeasureCount, MSB, LSB, Pulses );
sprintf( Message4, "%u.%03u Meters", Whole, Partial );
}

void Init()
{
cli();
strcpy( (char*)Message4, "Started Hello World");
DDRB = 0;  // All pins input
DDRB |= 1;
PORTB &= ~BlinkPin; // Make pins low to start

// turns on pin change interrupts
PCICR |= 1;
//
DDRB |= PulsePin; // Pulse pin output
PORTB  |= EchoPin;    // Enable pull up
PCMSK0 |= EchoPin;    // turn on interrupts on pins

sei();

}
u8g_t u8g;

void u8g_setup(void)
{
u8g_InitI2C(&u8g, &u8g_dev_ssd1306_128x64_i2c, U8G_I2C_OPT_NONE);
u8g_Begin( &u8g );
}
void StartMeasure()
{
cli();
PORTB |= PulsePin;
_delay_us( 11 );
PORTB & = ~PulsePin;
// I used a small delay here, sometimes it seemed to immediately start measuring, maybe it picked up the pulse I sent
// So wait just a little bit
_delay_us( 2 );
sei();
}
void draw()
{
u8g_SetFont(&u8g, u8g_font_6x10);

u8g_DrawStr(&u8g, 0, 10, Message0 );
u8g_DrawStr(&u8g, 0, 21, Message1 );
u8g_DrawStr(&u8g, 0, 32, Message2 );
u8g_DrawStr(&u8g, 0, 43, Message3 );
u8g_DrawStr(&u8g, 0, 54, Message4 );
}

int main(void)
{
Init();
u8g_setup();

strcpy( Message4, "Hello world" );

StartMeasure();
u8g_FirstPage(&u8g);
while(1)
{
do
{

draw();
}
while ( u8g_NextPage( &u8g ) );
}
}

}



### Galaxy S3 Compass Incorrect Values

Sunday, February 7th, 2016

Hmph, spent an afternoon debugging my Galaxy S3 Sensor app since it stopped working, checked all code changes for the last year then checked all the transformations etc. until I realized the compass sensor itself gave invalid values.

As a reminder to myself. Seems that you sometimes need to calibrate it:

1. Phone
2. *#0*#
3. In the test menu press sensors
4. Immediately start rotating the phone around all its axes until the little compass at the bottom of the screen shows 3.
5. Then lay it flat on the table and check that it displays the correct values (comparing it with a real compass) while rotating it over 360°
6. If not correct go back to the sensor menu and repeat from 3 onwards.

### Controlling 1306 128×64 OLED Display with an ATMega328P

Wednesday, February 3rd, 2016

Note that I mostly build on what other people researched and just put those results together however it is nice to have everything in one place. All credits go to the various authors I link to except for the glue in between (i.e. Atmel Studio part). So this will be mainly a starting point to the various sites.

This is a lot of text and I probably will spice it up a bit in the future with pictures of scantily dressed women, I mean boring diagrams and photos of ATMega328p’s.

Ok, with that out of the way let’s start. First off what do we need:

1. Raspberry Pi
2. ATMega328P
4. Wires to connect stuff
5. Optional 5 * 1K resistors (to protect your Raspberry Pi).
6. And of course an OLED display
7. Windows (I do it on windows you have to improvise for Linux however most or almost all tutorial are for Linux so you should be OK there).
8. Atmel Studio

ATMega328P’s are fond of breadboards and easily trapped in one. No worries though, I added a life support system to feed it.

Setting up and connecting the ATMega328P to your RPi

For that go to this site BUT instead of using the pin assignment for the ATTiny go to this site. Admire the ATMega328P with its many pins for a while and in the mean time I am going to get a cup of coffee while you  lookup and connect the corresponding pins for the ATMega328P.

So you’re finally back (I drank coffee, cleaned the house and destroyed some IC’s with static electricity in the mean time but that’s OK).

Atmel Studio

We will be using Atmel Studio to compile stuff and then scp the resulting binary to your Raspberry.

1. Download Atmel Studio. You probably have to supply your email and you get some nice ‘Maker’ info from Atmel. Your are now an official ‘Maker’ 🙂
2. Install Atmel Studio. This is Visual Studio under the hood so if you are familiar with it you feel right at home. If not and haven’t used any IDE before, you are probably looking at a confusing quagmire of User Interface Elements. Don’t worry though, I’ve got you covered, even if you haven’t got the remotest clue what you’re doing or even what we’re trying to accomplish here in the first place.
3. Start it and after a while you arrive at the welcome page.
4. Select ‘New Project…
5. Select ‘GCC Executable Project’
6. Give it an original name (like OLED).
7. Now there is a device selection window (may not be the top window). Click on it and select  ATMega328P
8. You are in your little project but as good cut & paste developers we are not going to program, yet…
9. Instead, of writing the lower layers, I downloaded the UG8 library for AVR and build on the Herculean job other developers have done.
11. In another window go into your Atmel studio source dir (one deeper than where your *.atsln file is). So if you named the project OLED you should select that dir twice to get there.
12. Select all ‘.c’ and ‘.h’ files and copy them to from ug8lib/src and copy them to the second window.
13. Now in Atmel studio the right pane should contain ‘Solution Explorer’ (if you didn’t mess with the windows).
14. Now right click on the project (not the solution which is the topmost node) and select ‘Add’->’Add Existing Item…’
15. Select all ‘.c’ and ‘.h’ files that you copied there to add them to the project.
16. Go to Build->Configuration Manager and select ‘Release’ as active build.

Almost there, our project is pretty much ready but we need to correct settings.

1. Again in the ‘Solution Explorer’ (the right window). Right click again on the project node (still not the top node) and select properties.
2. Basically if you look in the ug8lib dir there was a Makefile and I just set all the options according to that (if I could find them).
3. Build->Configuration->Select Release
4. Select ‘Toolchain’
5. Select the following options for Avr Gnu Compiler:
6. General: -mcall-prologues
7. General: -funsigned-char
8. General: -funsigned-bitfields
9. Preprocessor: Unseselect everything (should already be unselected)
10. Symbols: Click on the green + symbol and add ‘F_CPU=8000000’ this is your processor speed 8MHz.
11. Optimization: Select Optimize for size (-Os) in the drop-down list
12. Optimization: Select everything except -mshort-calls
13. Warnings: Select -Wall
15. General: -WL  (actually not sure if it is an L or the number 1)
16. Optimization: -WL –gc-sections

Then Build->Clean Build (not really necessary probably) followed by Build->Rebuild All. Now pray to your favorite God. I pray to the god of Jesus but you can pray to yours and we will see who’s build succeed :-).

If there are almost no errors. I got a warning about a pointer that was incompatible, however when I searched the source code for that, it was because they defined a function pointer as taking a parameter as ‘void *’ instead of the pointer to the function you assign it to, is using. Since they were the same size it isn’t a problem.

Now go into the OLED dir where you copied the .c and .h file and there should be a directory ‘Release’. If there isn’t but there is a ‘Debug’ instead you didn’t select the right project build in Build->Configuration Manager.

In that directory there should be a file ending in *.hex and that file is your program and need to be copied to your Raspberry Pi.

Modify the Raspberry Pi project for your hex file

Now in the source I assume the clock is 8MHz (#define F_CPU  8000000) so we must ‘fuse‘ the ATMega to 8MHz (i.e. turn of the clock divider):

sudo /usr/local/bin/avrdude -p m328p -P /dev/spidev0.0 -c linuxspi -b 10000 -U lfuse:w:0xE2:m

When you went through the instruction on this site, you ended up with a blinky program in the blinky directory.

cp -R that blinky dir to an OLED dir and instead of the Makefile we can use the following script (I just looked at the Makefile from this  site so all credits go to the original author ):

sudo gpio -g mode 22 out
sudo gpio -g write 22 0
sudo /usr/local/bin/avrdude -p m328p -P /dev/spidev0.0 -c linuxspi -b 10000 -U flash:w:OLED.hex
sudo gpio -g write 22 1

By running the above commands you flash your program to the ATMega328p.

It should write and verify just as with the blinky app.

Hooking up the Display

With the display in front of you the pin assignment is GND, VCC, CLCK, DATA. Make sure to flip it vertically so they stay in the same order now connect them as follows:

• GND connect to the GND of you breadboard
• Make sure you use the 3.3 V of your RPi not the 5V.
• VCC connect to the 3.3V
• CLK connect to the upper right pin (assuming pin 1 (with the notch) is at the top left) of the ATMega328p i.e. pin 28 top right pin
• DATA connect to the second pin at the upper right (i.e. the pin just below where we connected CLK) i.e. pin 27

Grand Finale

Connect the reset pin i.e. the upper pin at the left (the one with the notch) to Gnd and then to the VCC to reset the device.

The device should now display ‘Hello world’. In the picture below it shows ‘Does it work?’ because I attached it to my RF433 receiver (and I sent that string from my RPi).

Final result. The (pretty bad) signal is from my RF433Mhz receiver. My RPi sends a string using a RF433 transmitter which is then received and decoded by an assembly routine I wrote for the ATMega328.  The resulting string is then displayed instead of the standard ‘Hello world’ string. Admittedly currently the string is only correctly decoded once in several times due to noise on the wires.

### wpa_supplicant Excessive Logging on Raspberry Pi

Friday, December 25th, 2015

Today I noticed there was excessive logging from wpa_supplicant on my RPI. Normally this is of course not a problem but if you run it on an SDC like I do it might wear the memory since there is a low limit on the number of times you can rewrite it (in comparison with a harddisk).

I think there is a parameter $IF_WPA_VERBOSITY that can be set somewhere (maybe in the network file) however I was not in the mood to spend time on it so I just changed the following code in /etc/wpa_supplicant/functions.sh  WPA_SUP_OPTIONS="-s -B -P$WPA_SUP_PIDFILE -i $WPA_IFACE"   WPA_SUP_OPTIONS="-qq -f /dev/null -s -B -P$WPA_SUP_PIDFILE -i $WPA_IFACE"  After that is seemed to have stopped. Need to do it again after an update/upgrade though. ### Controlling Pump Sunday, November 22nd, 2015 Here a simple schematic to control a pump. I haven’t tested it with RPi’s GPIO output. I just connected the resistor to 12V. However I would expect it to work but you probably have to divide the resistor by 4. You want the resistor low enough to saturate the transistor, a BDX53B Darlington transistor, so voltage drop and power dissipation will be minimal. I bought the pump on banggood.com for 10€ number SKU235205. The BDX53B should be around 1€. It needs a high start current and then about 300mA for continues operation. However you also want it to be as high as possible to minimize the load on your RPi. I calculated the resistor to be 100k however that didn’t get the pump started so I lowered it tenfold. The input current is still very low and well within specs for both RPi as well as the transistor’s maximum base current. I didn’t add a diode in parallel to the pump since the BDX53B already has one internally which can withstand 8A forward current, which should be enough. Calculation are as follows: • Measured pump current 300mA • In the sheet for the BDX53B the gain for Ic for 300 mA is about 2000. • Therefore the current through the Base would be 300 mA / 2000 = 150 µA • R1 = Vcc – Vbe (from BDX53B spec) / Ibase = ( 12 V – 1.3 V ) / 150 µA = 75k Ω However to get the pump started we actually need more like 3 A (I could not measure it exactly it was just a blink on my multimeter) • Measured pump current approximately 3A • In the sheet for the BDX53B the gain for Ic for 3A is about 5000. • Therefore the current through the Base would be 3A / 5000 = 600 µA • R1 = Vcc – Vbe (from BDX53B spec) / Ibase = ( 12 V – 2 V ) / 600 µA = 16 kΩ (I used 10 kΩ but 15 kΩ should do) • For RPi (untested!) ( 3.3 – 2 ) / 600 µA = 2166 Ω (so a 2.2 kΩ resistor should work). Would the RPi get damaged it the transistor breaks down and shorts R1 to ground: • 3.3 V / 2000 Ω (make sure to take error in resistor into account therefore I use a lower value) = 1.6 mA The RPi can easily deliver 1.6 mA (assuming no other ports are connected). So if the transistor breaks down it will not harm your RPi. As always use at your own risk and verify that my calculations are correct before using it. ### Math MediaWiki extension with Texvc problem. Thursday, May 28th, 2015 This puzzled me for a while. My math was rendered as$ Hard to understand math goes here $instead of a nice png. I don’t fully understand the problem but made it work anyway by hacking the Math extension sources. First to make it work texvc must be installed (obviously). You must be able to run it as your web server (whatever that account is running on). So far the obvious part. Now you have to add the following to your LocalSettings.php in the root wiki dir in this order: $wgMathValidModes[] = array( MW_MATH_MATHJAX, MW_MATH_PNG, MW_MATH_SOURCE ); // Define MathJax as one of the valid math rendering modes $wgUseMathJax = false; // Enable MathJax as a math rendering option for users to pick$wgDefaultUserOptions['math'] = MW_MATH_SOURCE; // Set MathJax as the default rendering option for all users (optional) $wgMathDisableTexFilter = true; // or compile "texvccheck"$wgUseTeX = true; $wgTexvc = '/usr/bin/texvc';$wgUploadDirectory= "upload"; $wgUploadPath = "upload";$wgMathPath = "{$wgUploadPath}/math";$wgMathDirectory = "{$wgUploadDirectory}/math";$wgTmpDirectory = "{$wgUploadDirectory}/tmp";$wgShowSQLErrors = 1; require_once "$IP/extensions/Math/Math.php"; Make sure the directories are created under your root wiki install so if the LocalSettings.php are in /home/wiki/ then there should be a dir • /home/wiki/upload • /home/wiki/upload/math and • /home/wiki/math/tmp (you can of course use other locations to make it more difficult to get it working 🙂 ). Now for some reason the tmpDir ad wgTexvc don’t make it into the following file (don’t ask me why) or maybe tmpDir is not defined altogether. So I just hardcoded them (don’t forget to redo it if it works and you do an upgrade some time in the future). Update extensions/Math/MathTexvc.php as follows:  public function callTexvc() { global$wgTexvc, $wgTexvcBackgroundColor,$wgHooks; $wgTexvc="/usr/bin/texvc"; wfProfileIn( __METHOD__ );$tmpDir = wfTempDir(); if ( !is_executable( $wgTexvc ) ) { wfDebugLog( 'texvc', "$wgTexvc does not exist or is not executable." ); wfProfileOut( __METHOD__ ); return $this->getError( 'math_notexvc' ); }$tmpDir="/tmp";

Then finally they appeared in the preview. But unfortunately my wiki is not in the root of the website and it didn’t use the complete URL in the final pages as it does in the preview. This can be solved as follows in MathTexvc.php. I just hardcoded (again) the prefix.

return Xml::element( 'img', $this->getAttributes( 'img',$attributes, array( 'src' => "/mydir/mywiki/".\$url

After that my png’s appear nicely in my wiki, great.

Hopes this helps someone out there.

### Shrink partition in xen guest

Saturday, April 25th, 2015

In a previous post I discussed extending a partition in a linux guest. Since I never can make up my mind, this post discusses the opposite, i.e. shrinking a paritition in a disk (again as opposed to a partition) in a logical volume. Since we do not have just one partition in the logical volume we have to delete and recreate.

The procedure is a little different since it is too large to make a backup in the other quest I do it on the xen host:

• Create a logical volume the size of the one we are going to rearrange for backup
• Mount the disk in the Logical Volume (LV) on another guest.
• In the other guest use fdisk to recreate them with the right size

lvcreate -L 100G -n lv_vm_tmp /dev//


For instance:

lvcreate -L 100G -n lv_vm_ishpublic_tmp /dev/nebulus-vg


Backup the original volume to the temp volume we just created (double check that the original is first and the backup is second).

cp /dev// /dev//

cp /dev/nebulus-vg/lv_vm_ishpublic /dev/nebulus-vg/lv_vm_ishpublic_tmp


Now add this to your ‘utility’ host (we are going to mount the volume there to resize). In your xen config for utility add the disk. For instance my volume group is ‘nebulus-vg’ and the disk with partitions I want to resize is ‘lv_vm_ishpublic’.

disk = [ '/dev/nebulus-vg/lv_sirius_ubuntu,raw,xvda,rw', '/dev/nebulus-vg/lv_vm_ishpublic,raw,xvdb,rw' ]


Then:

Print the partitions on xvdb

root@sirius:~# fdisk /dev/xvdb

Welcome to fdisk (util-linux 2.25.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/xvdb: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x76ff0726

Device     Boot     Start       End   Sectors Size Id Type
/dev/xvdb1           2048 201328639 201326592  96G 83 Linux
/dev/xvdb2      201330686 209713151   8382466   4G  5 Extended
/dev/xvdb5      201330688 209713151   8382464   4G 82 Linux swap / Solaris

Partition 3 does not start on physical sector boundary.

Command (m for help): q


Run a file system check:

e2fsck -f /dev/xvdb1


Try resizing the partition (we can pass a very low number to resize2fs so it will print the minimum we need:

resize2fs /dev/xvdb1 1G
resize2fs 1.42.10 (18-May-2014)
resize2fs: New size smaller than minimum (4468729)


You probably do not want to have the exact minimum, I use approx. double the size.

resize2fs /dev/xvdb1 8388608
resize2fs 1.42.10 (18-May-2014)
Resizing the filesystem on /dev/xvdb1 to 8388608 (4k) blocks.
The filesystem on /dev/xvdb1 is now 8388608 blocks long.


Though the filesystem is resized the partition is still the original size. We need to modify the partition table on the disk where the new end is. Calculate this from the output of resize2fs and the size of a sector from fdisk (sector size was 512 in my case, make sure you substitute the values you got!).

• New filesystem size (output from resize2fs) = 8388608 (4k) blocks = 8388608 * 4 * 1024 = 34359738368 bytes
• New filesystem size in sectors = 34359738368 / 512 = 67108864 sectors (if you get a fractional value round it up)
• Therefore the end sector in fdisk will be start sector + size = 2048 + 67108864 = 67110912
• I also want a 2G swap partition i.e. 2 * 1024 * 1024 * 1024 / 512 = 4194304 sectors
• Therefore the end of the disk will be at 67110912 + 4096 = 71305217 sectors
 fdisk /dev/xvdb

Welcome to fdisk (util-linux 2.25.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/xvdb: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x76ff0726

Device     Boot     Start       End   Sectors Size Id Type
/dev/xvdb1           2048 201328639 201326592  96G 83 Linux
/dev/xvdb2      201330686 209713151   8382466   4G  5 Extended
/dev/xvdb5      201330688 209713151   8382464   4G 82 Linux swap / Solaris

Partition 3 does not start on physical sector boundary.

Command (m for help): d
Partition number (1,2,5, default 5):

Partition 5 has been deleted.

Command (m for help): d
Partition number (1,2, default 2):

Partition 2 has been deleted.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.


Create the new partitions:

Command (m for help): n Partition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions) Select (default p): Using default response p. Partition number (1-4, default 1): First sector (2048-209715199, default 2048): Last sector, +sectors or +size{K,M,G,T,P} (2048-209715199, default 209715199): 67110912 Created a new partition 1 of type 'Linux' and of size 32 GiB. Command (m for help): n Partition type p primary (1 primary, 0 extended, 3 free) e extended (container for logical partitions) Select (default p): Using default response p. Partition number (2-4, default 2): First sector (67110913-209715199, default 67112960): 67110913 Last sector, +sectors or +size{K,M,G,T,P} (67110913-209715199, default 209715199): 71305217 Created a new partition 2 of type 'Linux' and of size 2 GiB.

Toggle the filesystem type on the swap partition:

t 2 82

Write the table back to disk, quit and shutdown the utility system:

w q shutdown -h now

Back on the xen host we now need to reduce the logical volume size. First we need to know the physical extent:

 vgdisplay
--- Volume group ---
VG Name               nebulus-vg
System ID
Format                lvm2
VG Status             resizable
MAX LV                0
Cur LV                7
Open LV               5
Max PV                0
Cur PV                1
Act PV                1
VG Size               3.64 TiB
PE Size               4.00 MiB
Total PE              953672
Alloc PE / Size       129825 / 507.13 GiB
Free  PE / Size       823847 / 3.14 TiB
VG UUID               tm4Q7z-LWIL-OUW3-egF0-8UP4-d2MJ-fW1sXX



PE Size is 4 MB

To calculate the new size in extends take the size in bytes from our last sector ( I am not sure if sectors start from 1 or 0, better be safe) from fdisk and add one.

• 71305217 + 1 = 71305218

Round up to the next vg extent boundary:

• RoundUp( Number of sectors * sectors size / PE Size )
• 71305218 * 512 / ( 4 * 1024 * 1024 ) = 8704.25
• RoundUp( 8704.25 ) = 8705

Shrink the logical volume

lvreduce -l 8705 /dev/nebulus-vg/lv_vm_ishpublic
WARNING: Reducing active logical volume to 34.00 GiB
THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce lv_vm_ishpublic? [y/n]: y
Reducing logical volume lv_vm_ishpublic to 34.00 GiB
Logical volume lv_vm_ishpublic successfully resized


And we are done. Well actually you would want to startup your utility VM again, do the mount of the filesystem and run e2fsck (see above) again to make sure that everything worked.

e2fsck /dev/xvdb1
e2fsck 1.42.10 (18-May-2014)
/dev/xvdb1: recovering journal
Setting free inodes count to 1775009 (was 1775029)
Setting free blocks count to 4183513 (was 4183757)
/dev/xvdb1: clean, 322143/2097152 files, 4205095/8388608 blocks


Bonus points enable swap.

If you configured a swap parition check it is working:

swapon -s


If not you have to add the swap partition. Let’s say it was on xvdb2 when you configured it in util then it probably is in xvda2 (make sure you get it right i.e. check if it exists in /dev isn’t mounted df -k). In the following I assume it is on /dev/xvda2

Now for ubuntu execute the following:

blkid /dev/xvda2


Copy the UUID and open /etc/fstab. Copy the swap line and change it as follows (use the printed UUID instead of the one below).

UUID=7340ea5f-3325-466a-8e2f-52c1cf15ae94 / ext4 errors=remount-ro 0 1
# swap was on /dev/xvda5 during installation
#UUID=75ecaea3-ff0d-4f90-b5e2-ea757bde5399 none swap sw 0 0
UUID=582dae7a-5d2a-4b4c-9685-993fc8454200 /dev/xvda2 swap sw 0 0

shutdown -r now


Log back in:

swapon -s
Filename                                Type            Size    Used    Priority
/dev/xvda2                              partition       1046524 0       -1


If satisfied don’t forget to start Util and cleanup the backups created.

### Extend partition in disk in xen linux guest

Sunday, April 12th, 2015

After searching the web for a method to resize a partition in a disk in a logical volume as opposed to just a partition in a logical volume (notice the difference ‘partition in’ or ‘disk in’). I could not find a method however if you have enough space you could use the following method.

Note that I used this to resize the partition of a virtual machine running in a logical volume (it should work with physical stuff too but then you might have to put a disk in another physical host or mount it using a live cd).

Util is a machine with the same version of the OS. tbe is the one we want to extend.

From a high level the procedure is as follows:

• Mount the disk with partitions you want to extend on another virtual machine (VM)
• Use cp to backup the disks
• Use fdisk to extend the partitions
• Use cp to restore the backup to the resized partitions
• On the VM with, now, resized disks, run resize2fs
• Bonus; resize and or add the swap partition

Everything is done as root. Prefix with sudo if you want or need to.

xl shutdown util
xl shutdown tbe


Extend the disk on the host.

vgs
lvs
lvextend -L8G /dev/<vg>/<lvm>


Unfortunately although the logical volume is extended the disk inside isn’t using the extra size. Since we do not simply mount the disk but have a complete disk within the lv we cannot simply use resize something. Instead we have to mount it on, preferably the same version of the OS and then make a backup of the partition(s). Drop everyting and create new partitions then copy it back:

edit util .cfg

vgs
lvs
disk = [ '/dev/<VG>/<lv util>,raw,xvda,rw', '/dev/<VG>/<lv tbe>,raw,xvdb,rw' ]


Bring our util system backup

xl create util.cfg


ls -tarl /dev/xvdb*


Make a backup of all the regular partitions (no need to backup swap of course).

mkdir /backup
# Repeat for all other partitions (you could skip swap if you want but I would be on the safe side and backup all just in case you got the partition wrong)
cp /dev/xvdb1 /backup/xvdb1backup1
cp /dev/xvdb2 /backup/xvdb1backup2
# etc..

fdisk /dev/xvdb

Command (m for help): p
Disk /dev/xvdb: 14 GiB, 15032385536 bytes, 29360128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x20230da1

Device Boot Start End Sectors Size Id Type
/dev/xvdb1 2048 8388607 8386560 4G 83 Linux
/dev/xvdb2 8388608 16777215 8388608 4G 82 Linux swap / Solaris



Now remove all the partitions listed (you didn’t skip the backup step, right…).

Command (m for help): d
Partition number (1,2, default 2):

Partition 2 has been deleted.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.


Then create new partitions make sure specify the size you want. The calculation is as follows:

Let’s say we want two partitions.

1. Primary partition 13GB
2. Swap partition remainder of the disk.

For the start value we use the start value that fdisk will print when pressing n, p. The number we have to add to this to get a 13GB partition is as follows. The first partition must be a primary partition. The following can be extended (if you need more than 3 other partitions) or more primary ones. The sector size is printed by fdisk (see above):

Diff = PartitionSizeInGB * 1024 * 1024 *1024 / SectorSizeInBytes

For instance

Diff = 13 * 1024 * 1024 * 1024 / 512 = 27262976

Therefore the last sector is:

LastSector = 27262976 + 2048 = 27265024

Partition type
p   primary (0 primary, 0 extended, 4 free)
e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-29360127, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-29360127, default 29360127): 27265024

Created a new partition 1 of type 'Linux' and of size 13 GiB.


And lastly create a swap partition using the rest of the disk i.e. just go with the default values.

Command (m for help): n
Partition type
p   primary (1 primary, 0 extended, 3 free)
e   extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2):
First sector (27265025-29360127, default 27267072):
Last sector, +sectors or +size{K,M,G,T,P} (27267072-29360127, default 29360127):

Created a new partition 2 of type 'Linux' and of size 1022 MiB.


We now have to change the type to linux swap which has code 82.

Command (m for help): t
Partition number (1,2, default 2):
Hex code (type L to list all codes): 82

Changed type of partition 'Linux' to 'Linux swap / Solaris'.

Command (m for help):


Write the results to disk:

w


Copy the backup back to the disk

cp /backup/xvdb1backup1 /xvdb1
#etc for all partitions


Shutdown util

shutdown -h now


In the host remove the disk again from util.cfg Now start the tbe

xl create tbe.cfg


Logon to tbe

df -k


Still no luck huh? 😉

su - root
resize2fs /dev/<the device you extended>

df -k


Should now list the new size.

If you configured a swap parition check it is working:

swapon -s


If not you have to add the swap partition. Let’s say it was on xvdb2 when you configured it in util then it probably is in xvda2 (make sure you get it right i.e. check if it exists in /dev isn’t mounted df -k). In the following I assume it is on /dev/xvda2

Now for ubuntu execute the following:

blkid /dev/xvda2


Copy the UUID and open /etc/fstab. Copy the swap line and change it as follows (use the printed UUID instead of the one below).

UUID=7340ea5f-3325-466a-8e2f-52c1cf15ae94 / ext4 errors=remount-ro 0 1
# swap was on /dev/xvda5 during installation
#UUID=75ecaea3-ff0d-4f90-b5e2-ea757bde5399 none swap sw 0 0
UUID=582dae7a-5d2a-4b4c-9685-993fc8454200 /dev/xvda2 swap sw 0 0

shutdown -r now


Log back in:

swapon -s
Filename                                Type            Size    Used    Priority
/dev/xvda2                              partition       1046524 0       -1


If satisfied don’t forget to start Util and cleanup the backups created.