What i did was i created some txt file and some code to index the file and have tried to find out in practice how lucene calculates relevance using the DefaultSimilarity class. Here is the file and the source code.

file: file_a.txt
jayant project manager project leader team leader java linux c c++ lucene apache solaris aix unix minix gnome kde ubuntu redhat fedora rpm deb media player vlc evolution exchange microsoft java vb vc vc++ php mysql java

source code: FIndexer.java
import java.io.*;
import java.util.Date;
import org.apache.lucene.index.*;
import org.apache.lucene.document.*;
import org.apache.lucene.analysis.*;public class FIndexer
{

public static void main (String[] args) throws Exception
{
String src_path = args[0];
String des_path = args[1];

File f_src = new File(src_path);
File f_des = new File(des_path);

if(!f_src.isDirectory() || !f_des.isDirectory())
{
System.out.println(“Error : “+f_src+” || “+f_des+” is not directory”);
System.exit(1);
}

IndexWriter writer = new IndexWriter(f_des,new WhitespaceAnalyzer(), true);

File[] files = f_src.listFiles();
for(int x=0; x < files.length; x++)
{
Document doc = new Document();
if(files[x].isFile())
{
BufferedReader br = new BufferedReader(new FileReader(files[x]));
StringBuffer content = new StringBuffer();
String line = null;
while( (line = br.readLine()) != null)
{
content.append(line).append(“\n”);
}

Field f1 = new Field(“name”,files[x].getName(), Field.Store.YES, Field.Index.NO);
doc.add(f1);
Field f2 = new Field(“content”, content.toString(), Field.Store.NO, Field.Index.TOKENIZED);
doc.add(f2);
Field f3 = new Field(“content2″, content.toString(), Field.Store.NO, Field.Index.TOKENIZED);
doc.add(f3);

}
writer.addDocument(doc);
}
writer.optimize();
writer.close();
}
}

I created copies of the file_a.txt as file_b.txt and file_c.txt and edited file_c.txt. Such that file_a.txt and file_b.txt have the same content (that is word java occurs 3 times in both the files). And file_c.txt has word java occuring only 2 times.

I created an index using FIndexer.java and used luke to fire queries on the index.

Firstly did a search content:java. And as expected i got 3 results with the following score.

Lets take the score for file_b.txt and see how it is calculated. The score is a product of
tf = 1.7321 (java occurs 3 times)
idf = 0.7123 (document freq = 3)

And the score of file_c.txt is a product of
tf = 1.4142 (java occurs 2 times)
idf = 0.7123 (document freq = 3)

Here score is equivalent to the fieldWeight since there is just one field. If more than one fields are used in the query, then the score would be a product of the queryWeight and fieldWeight

So if i change the query to Content:java^5 Content2:java^2. Here i am boosting java in content by 5 and java in content2 by 2. That is java in content is 2.5 times more important than java in content2. Lets check the scores.

Lets look at how the score was calculated

Again for file_b.txt
0.2506 = 0.1709 (weight of content:java^5) * 0.0716 (weight of content2:java^2)Out of which Weight of content:java^5 is
= 0.9285(Query weight) [ 5 (boost) * 0.7123 (idf docFreq=3) * 0.2607 (queryNorm) ]
* 0.1928(field weight) [ 1.7321 (tf=3) * 0.7123 (idf docFreq=3) * 0.1562 (fieldNorm) ]

And weight of content2:java^2 is
= 0.3714(Query weight) [ 2 (boost) * 0.7123 (idf docFreq=3) * 0.2607 (queryNorm) ]
* 0.1928(Field weight) [ 1.7321 (tf=3) * 0.7123 (idf docFreq=3) * 0.1562 (fieldNorm) ]The same formula is used for calculating of score of file_c.txt except for the fact that termfrequency = 1.4142 (tf of content:java or content2:java is 2)

This explains a little about scoring in lucene. The scoring can be altered by either changing the DefaultSimilarity class or extending the DefaultSimilarity and changing certain factors/calculations in it. More on how to change the scoring formula later.

1. firstly get the available splash screens. Do a

sudo apt-get install usplash*

It will display a list of available splash screens with ubuntu.

2. Next check out the available splash screens in /usr/lib/usplash

ls -lh /usr/lib/usplash

total 12M
-rw-r–r– 1 root root 43K 2006-11-23 17:42 debian-edu-usplash.so
-rw-r–r– 1 root root 2.3M 2007-03-30 17:33 edubuntu-splash.so
lrwxrwxrwx 1 root root 36 2007-05-17 23:20 usplash-artwork.so -> /etc/alternatives/usplash-artwork.so
-rw-r–r– 1 root root 2.0M 2006-10-17 15:13 usplash-theme-ichthux.so
-rw-r–r– 1 root root 2.3M 2007-04-07 15:36 usplash-theme-kubuntu.so
-rw-r–r– 1 root root 2.6M 2007-04-10 18:28 usplash-theme-ubuntu.so
-rw-r–r– 1 root root 2.0M 2007-03-19 16:17 usplash-theme-xubuntu.so

Here you can see that you have 6 splash screens and one soft link. Check out the softlink.

ls -lh /etc/alternatives/usplash-artwork.so

lrwxrwxrwx 1 root root 41 2007-06-29 08:38 /etc/alternatives/usplash-artwork.so -> /usr/lib/usplash/usplash-theme-xubuntu.so

It points back to one of the screens from the /usr/lib/usplash directory. So to change the screen simply change the soft link

sudo ln -sf /usr/lib/usplash-theme-kubuntu.so /etc/alternatives/usplash-artwork.so

And now check the new link

3. reconfigure the linux image

sudo dpkg-reconfigure linux-image-

Running depmod.
update-initramfs: Generating /boot/initrd.img-2.6.20-16-generic
.
.
Updating /boot/grub/menu.lst … done

Thats done…

To check the new screen, no dont reboot simply type in

sudo usplash

And you can see the new screen – bingo. To get back to your xwindows environment press CTRL-ALT-F7

P.S.

here is the link for advanced users for creating their own splash screens
http://codeidol.com/unix/ubuntu/X11/Change-the-Ubuntu-Splash-Screen/
Anyone who develops his/her own splash screen can share it out with other ubuntians…

He used lucene-php in zend framework for building a lucene index and searching on the index, and was facing issues with search times. It turned out that mysql full text index was performing better than lucene index.

So i did a quick benchmark and found the following stuff

1. Indexing using php-lucene takes a huge amount of time as compared to java-lucene. I indexed 30000 records and the time it took was 1673 seconds. Optimization time was 210 seconds. Total time for index creation was 1883 seconds. Which is hell lot of time.

2. Index created using php-lucene is compatible to java-lucene. So index created by php-lucene can be read by java-lucene and vice versa.

3. Search in php-lucene is very slow as compared to java-lucene. The time for 100 searches are –

jayant@jayantbox:~/myprogs/java$ java searcher
Total : 30000 docs
t2-t1 : 231 milliseconds

jayant@jayantbox:~/myprogs/php$ php -q searcher.php
Total 30000 docs
total time : 15 seconds

So i thought that maybe php would be retrieving the documents upfront. And changed the code to extract all documents in php and java. Still the time for 100 searches were -

jayant@jayantbox:~/myprogs/java$ java searcher
Total : 30000 docs
t2-t1 : 2128 milliseconds

jayant@jayantbox:~/myprogs/php$ php -q searcher.php
Total 30000 docs
total time : 63 seconds

The code for php search for lucene index is:

/* *      searcher.php *      On 2007-06-06 * By jayant  * */

include("Zend/Search/Lucene.php");

$index = new Zend_Search_Lucene("/tmp/myindex");echo "Total ".$index->numDocs()." docs\n";$query = "java";$s = time();for($i=0; $i{ $hits = $index->find($query);// retrieve all documents. Comment this code if you dont want to retrieve documents foreach($hits as $hit)  $doc = $hit->getDocument();

}$total = time()-$s;echo "total time : $total s";?>

And the code for java search of lucene index is

/* *      searcher.java *      On 2007-06-06 * By jayant  * */

import org.apache.lucene.search.*;import org.apache.lucene.queryParser.*;import org.apache.lucene.analysis.*;import org.apache.lucene.analysis.standard.*;import org.apache.lucene.document.*;

public class searcher {

 public static void main (String args[]) throws Exception {  IndexSearcher s = new IndexSearcher("/tmp/myindex");  System.out.println("Total : "+s.maxDoc()+" docs");  QueryParser q = new QueryParser("content",new StandardAnalyzer());  Query qry = q.parse("java");

  long t1 = System.currentTimeMillis();  for(int x=0; x  {   Hits h = s.search(qry);// retrieve all documents. Comment this code if you dont want to retrieve documents   for(int y=0; y   {    Document d = h.doc(y);   }

  }  long t2 = System.currentTimeMillis();  System.out.println("t2-t1 : "+(t2-t1)+" ms"); }}

Hope i havent missed anything here.

1. Insert the card and look at dmesg
output : 0.0: ttyS3 at I/O 0x2e8 (irq = 3) is a 16C950/954

This says that your card was detected properly.
You can also run “pccardctl info”
output =
PRODID_1=”CDMA1X”
PRODID_2=”CARD”
PRODID_3=”"
PRODID_4=”"
MANFID=0279,950b
FUNCID=2
PRODID_1=”"
PRODID_2=”"
PRODID_3=”"
PRODID_4=”"
MANFID=0000,0000
FUNCID=255

2. edit /etc/wvdial.conf

[Dialer Defaults]
Modem = /dev/ttyS3
Baud = 57600
SetVolume = 0
Dial-AT-OK ATDT Command =
Init1 = ATZ
FlowControl = Hardware (CRTSCTS)
Phone = #777
Username =
Password =
New PPPD = yes
Carrier Check = no
Stupid Mode = 1

3. Run set serial
(download and install setserial – if you dont have setserial)
“setserial /dev/ttyS3 baud_base 230400″

4. run wvdial
wvdial

So to get it running everytime you will need to create a shell script doing steps 3 & 4. Say mydial

—————
setserial /dev/ttyS3 baud_base 230400
wvdial
—————

PS
This method is also not perfect. I got one card working with this procedure, but could not get another card working using the same procedure.

Let me know if this helps you getting your ZTE MC 315 card running on linux.

PS 2
And to add to it, finally i have got my card running.
The trick is simple – you have to play with the UART and the baud_base.

Just run setserial on your machine and it will give the following output for UART and baud_base

* uart set UART type (none, 8250, 16450, 16550, 16550A,
16650, 16650V2, 16750, 16850, 16950, 16954)
* baud_base set base baud rate (CLOCK_FREQ / 16)

What i did was that i kept on changing my UART and kept the baud_base as 230400. I changed my UART to 16850 and then to 16750. And my card responded at 16750. And i got connected to the internet

So, My dialup script does this before doing wvdial

setserial /dev/ttyS3 uart 16750
setserial /dev/ttyS3 baud_base 230400

And this gets my card running. So to get your card running, i would suggest you try setting different UART and baud_base…

When you occasionally have a really bad day, and you just need to take It out
on someone, don’t take it out on someone you know — take it out on someone
you don’t know. I was sitting at my desk when I remembered a phone call I had
forgotten to make. I found the number and dialed it.

A man answered, saying, “Hello.”

I politely said, “Could I please speak with Robin Carter?”

Suddenly, the phone was slammed down on me. I couldn’t believe that Anyone
could be so rude. I realized I had called the wrong number. I tracked down
Robin’s correct number and called her. I had accidentally transposed the last
two digits of her phone number. After hanging up with her, I decided to call
the ‘wrong’ number again.

When the same guy answered the phone, I yelled, “You’re an *******!” and hung
up.

I wrote his number down with the word ‘*******’ next to it, and put it in my
desk drawer.

Every couple of weeks, when I was paying bills or had a really bad day, I’d
call him up and yell, “You’re an *******!” It always cheered me up.

When Caller ID came to our area, I thought my therapeutic ‘*******’ calling
would have to stop. So, I called his number and said, “Hi, this is John Smith
from the Telephone Company. I’m just calling to see if you’re familiar with
the Caller ID program?”

He yelled, “NO!” and slammed the phone down.

I quickly called him back and said, “That’s because you’re an *******!”

One day I was at the store, getting ready to pull into a parking spot.
Some guy in a black BMW cut me off and pulled into the spot I had patiently
waited for. I hit the horn and yelled that I had been waiting for that spot.
The idiot ignored me. I noticed a “For Sale” sign in his car window . . so, I
wrote down his number.

A couple of days later, right after calling the first ******* ( I had his
number on speed dial), I thought I had better call the BMW *******, too.

I said, “Is this the man with the black BMW for sale?”

“Yes, it is.”

“Can you tell me where I can see it?”

“Yes, I live at 1802 West 34th Street . It’s a yellow house, and the car’s
parked right out in front.”

“What’s your name?” I asked.

“My name is Don Hansen,” he said.

“When’s a good time to catch you, Don?”

“I’m home every evening after five.”

“Listen, Don, can I tell you something?”

“Yes?”

“Don, you’re an *******.”

Then I hung up, and added his number to my speed dial, too. Now, when I had a
problem, I had two assholes to call.
But after several months of calling them, it wasn’t as enjoyable as it used to
be. So, I came up with an idea. I called ******* #1.

“Hello.”

“You’re an *******!” (But I didn’t hang up.)

“Are you still there?” he asked.

“Yeah,” I said.

“Stop calling me,” he screamed.

“Make me,” I said.

“Who are you?” he asked.

“My name is Don Hansen.”

“Yeah? Where do you live?”

“*******, I live at 1802 West 34th Street , a yellow house, with my black
Beamer parked in front.”

He said, “I’m coming over right now, Don. And you had better start saying your
prayers.”

I said, “Yeah, like I’m really scared, *******.”

Then I called ******* #2.

“Hello?” he said.

“Hello, *******,” I said.

He yelled, “If I ever find out who you are…!”

“You’ll what?” I said.

“I’ll kick your ass,” he exclaimed.

I answered, “Well, *******, here’s your chance. I’m coming over right now.”

Then I hung up and immediately called the police, saying that I lived at 1802
West 34th Street, and that I was on my way over there to kill my gay lover.

Then I called Channel 13 News about the gang war going down on West
34th Street.

I quickly got into my car and headed over to 34th street .

When I got there, I saw two assholes beating the crap out of each other
in front of six squad cars, a police helicopter, and the channel 13
news crew.

NOW, I feel better – This is “Anger Management” at its very best.

(origin) A –> B –> C –> D –> A (origin)

Each instance has its own serverid. A query originating from any machine will travel through the loop replicating data on all the machines are return back to the origniating server – which in effect will identify that the query had originated from here and would not execute/replicate the query further.

To handle auto_increment field, two variables are defined in the configuraion of the server.

1. auto_increment_increment : controls the increment between successive AUTO_INCREMENT values.

2. auto_increment_offset : determines the starting point of AUTO_INCREMENT columns.

Using these two variables, the auto generated values of auto_increment columns can be controlled.

Once the replication loop is up and running, any data inserted in any node would automatically propagate to all the other nodes.

ISSUES WITH MULTI-MASTER REPLICATION & AUTOMATIC FAILOVER

1. LATENCY

There is a definite latency between the first node and the last node. The replication steps lead to the slave reading the master’s binary log through the network and writing it to its own relay-bin log. It then executes the query and then writes it to its own binary log. So each node takes a small amount of time to execute and then propagate the query further.

For a 4 node multi-master replication when i replicated a single insert query containing one int and one varchar field, the time taken for data to reach the last node is 5 ms.
This latency would ofcourse depend on the following factors
a) amount of data to be relicated. Latency increases with increase in data
b) Network speed between the nodes. If the data is to replicated over internet, it will take more time as compared to that on nodes in LAN
c) Amount of indexes on the tables being replicated. As the number of indexes increase, the time taken to insert data in tables also increases. Increasig the latency.
d) The hardware of the machine and the load on the machine will also determine how fast the replication between master and slave will take place.

2. AUTOMATIC FAILOVER

Automatic failover can be accomplished using events and federated tables in mysql 5.1. Federated tables are created between master and slave and are used to check connection between master and slave. An event is used to trigger a query on the federated table which checks the connection between master and slave. If the query fails, then a stored procedure can be created which should chunk the master out of the replication loop.

So suppose in the loop shown above, A goes out, then the event on B would detect that A is out and would make D as a master of B.

This is the theoretical implementation of automatic failover. Practically there are a few issues with its implementation.

a) for failover, you need to know the position of master’s master from where the slave should take over. (So B should know the position on D from where it has to start replication). One of the ways to do this is that each slave logs its master’s position on a table which is replicated throughout the loop. But this again is not possible using events & stored procedures – cause there is no query which can capture the information available from “SHOW SLAVE STATUS” query in a variable and write it in a table. Another way to do this is to have an external script running at an interval of say 10 seconds which logs this information and checks the federated tables for any disruption in the master-slave connection. With this methodology, the problem is that there could be a 10 second window period during which data can be lost.

b) You could also land into an infinite replication situation. Lets look how using the 4 node example above. Suppose “A” goes down and It has a query in its binary log which has to be replicated throught the loop. So the query propagates from “B” to “C” and finally to “D”. Now since “A” is down and failover has happened, “B” would be the slave of “D”. So the query which originated from “A” would go from “D” to “B”. Thats because if “A” would have been there, it would have identified its own query and stopped it. But since “A” is out of the loop, the query will not be stopped and it will propagate to “B” and will always be in the loop. This can result in either the query running in the loop indefinitely or an error on all the slaves and all the slaves going down.

These are the major issues with multi-master replication and automatic failover. Though one can still live with the automatic failover scenario – as it might occur once in a while. But the latency between first and last node during replication has no solution. This latency is due to physical constraints and cannot be avoided.

A work around would be distributing the queries based on tables on all the nodes. So that queries for a table would always be served from a single node. But this then would result in table locks on that node and again we would not be able to reap the benefits of multi-master replication.

At indian railways, the trains are never expected to arrive on time. All trains run late. And if you are standing at one of the platforms, you will constantly hear – “train x which was supposed to come at time y is delayed by delta minutes/hours. The inconvenience caused is deeply regretted“. And the announcer would never sound as if he/she was regretting anything. It is said in a matter-of-fact way – as if we are supposed to know that the delay was naturally expected – just like you expect corruption in each and every section of the government of like you expect to get ketchup free with your samosa.

Let me brief you as to what happened last night. Had been to the delhi station to put my mom on the train for baroda. First of all – it had been a long time since i had been to the railways. So i had to enquire about where the parking was. And very few people seemed to know where it was. And since there is no visible board giving directions – i had to run here and there to get the location of parking space.

Well next – get information about the platform where the train would arrive. And the only way to get this information is to stand in queue – after about 100 people and wait your turn. The uncleji at the enquiry window – does not give a shit about who is on the other side of the window. I had time to enquire – what about people who arrived late and need the info – can they wait in queue for 30 minutes to enquire about which platform they should go. It seems the situation in delhi was worse. Its better in baroda – where there is less crowd and the information is properly displayed on the general information board.

Now – since i know which platform i have to go, i will have to spend rs 3/- to get a platform ticket – so that any Ticket Checker cannot catch me and ask for a 50/- or 100/- ghoos for travelling without a ticket. They usually do. There has been so many instances with my friends where the TC simply asked for a Rs 50/- note to let them go. And the way to avoid this hassle is to get a platform ticket. And to get a platform ticket is another hassle. On new delhi station, it is really difficult first to locate the window which gives platform ticket. Officially platform ticket should be available on all windows. But it seems due to scarcity – its there only on one window. And the queue in front of the window is huge. Hmm, there are always queues at all windows on indian railway stations. It seems that the indian railways is very low on manpower. Well actually thats how Laalu prasad yadav – our respected railway minister brought the operating cost of Indian railways to 78% – the best in the world.

And after getting the platform ticket, we proceeded to the platform. Wow, what a place. It seems that people like to live on the platform. Everywhere – there are people sitting, chatting, boozing, gambling. You have to look for space to put your foot to move forward. I think, this is what they mean by population explosion.

What happens in baroda station is that there is a display on the platform which says where each coach will come for the train. So it becomes easy to move and get placed right in front of the coach. But in Delhi – the capital of india, the numbers on the boards are misplaced so that you have to drag your luggage and run in between the crowded platform to your bogie.

And then the trains are always crowded. It is difficult to get a ticket in a train in indian railways. There is always a waiting list and extra people on the train for whom it is important to reach their destination, but were unable to get a reservation. There are people sleeping on the floor, outside the loo and sitting on your berths.

60 years of independence, and indian railways – it seems – stays where it was.

You: Achha! To usse Kya hoga…

You ‘re Colleague: Impression!! ! Appraisal!!! Har appraisal main tu No 1! Hike in salary!! Extra Stocks

You: Phir kya hoga…

Your Colleague: Project Leader ban jaayega..Phir Project Manager!!! Phir Business Manager! One day U will be a Director of the Company man !!

You: Acchha to phir kya hoga…

Your Colleague: Abe phir tu aish karega! Koi kaam nahin karna padega! Araam se office aayega aur MAIL check karega.

You: To ab main kya kar raha hoon????

“Dikhawe pe na jao, apni akal lagao.

Programming hai waste, trust only copy-paste “

Powered by ctrl C

Driven by ctrl V

It is same in relationships, you expect that your close ones grow up in life. That your wife takes care of you. And your wife would expect you to take care of her, of her emotions, of her will. You expect that your wife or your hubby has great personality in front of others. That others say that you two are a nice couple. You expect that you both listen to each other and respect each other. Etc.. Etc..

The expectations never end, and so neither does the issues or misunderstandings that come out of it.

And the best and simplest way to get out of this is “not to expect“. Wow, you would say – how is that possible. How can you not expect. How can you let go of your emotions. How can you say that “let things be as it is”.

It is the biggest thing to ask. But that is what the monks do. They just let go of their emotions. Lead a life where they dont care for anything. They meditate. Live with the bare minimum. And they are happy most of the times.

I dont know whether they believe in GOD or not. I dont. How can you believe something that you have not seen, not felt, not heard. Do you believe your friend if he says something odd. I wont. I will have to look at it to believe it. Oh, what would you say if i tell you that i met GOD yesterday. Would you believe me? No way… I think it is the same thing. People say that there is GOD, but where??

And then there is love. I think, i have written about it before. A feeling that cannot be defined. How can you expect something from someone you love. That is not love. That is possessiveness. You dont have expectations in love. All you intend is to give to the one you love. You take care of him/her. You forget and forgive all his/her mistakes. You want that person to be healthy. In all if you love someone, you would expect that person to be happy. And would do anything to make him/her happy. That i think is love.

So expectations dont come in between GOD(whom i dont believe in) and love. But in this materialistic world – you have to eat, you have to drink, and you have to live. And you have some materialistic/physical and emotional needs. And this is where expectations come in picture and they ruin everything.

Whenever you expect something from someone – you open a path to misunderstandings and issues.

Eventually everyone who is born has to die. Then why not make this miserable life happy for others and for self.