Soundex singed different values for same names? [SAS]

Prev: what is the main purpose of informat?
Next: Single Hurdle Poisson - MCMC Procedure

From: Sierra Information Services on 11 Mar 2010 17:12

I am glad a solution/understanding to this problem has already been
identified, but I wanted to point out that the SOUNDEX Function, and
the underlying SOUNDEX algorithim it implements, is a failry weak way
of trying to meausre similarity/dissimilarity between two text
strings. There are a lot of limitations to it, especially when using
it on family names.

You may want to explore use of the SPEDIS (spelling distance) ,
COMPGED (compute generalize distance) and COMPLEV (compute Levenshtein
edit distance) functions as more powerful tools for your project.
SPEDIS was added in V8 and the other two were added in SAS 9,0. You
can also use the CALL COMPCOST routine in SAS 9 to assign your own
"penalty costs" if you don't like the ones that are implemented by
default in the COMPGED function.

There are examples of how to use SOUNDEX, SPEDIS, COMPGED and COMPLEV
in the PDF of my paper "Becoming More FUNCTIONal in SAS 9 Software,"
available for free download at http://www.sierrainformation.com. From
the home page click on "Free Downloads" and take things from there.

Hope this helps

Andrew Karp
Sierra Information Services
http://www.sierrainformation.com

On Mar 11, 11:42ï¿½am, Nancy <nancy0...(a)gmail.com> wrote:
> Yes, that is the reason.
> I checked the code again.
> And found that I used the soudex function before I seperated the first
> name and middle for some names.
>
> Thank you very much!
>
> Xiaohong
>
> On Mar 11, 2:16ï¿½pm, "data _null_;" <datan...(a)gmail.com> wrote:
>
>
>
> > On Mar 11, 12:58ï¿½pm, Nancy <nancy0...(a)gmail.com> wrote:
>
> > > I ï¿½used the
>
> > > IDF=soundex(first_name)
>
> > > to get the soundex ID for the first name.
>
> > > Is there anything wrong?
>
> > > Thank you!
>
> > > On Mar 11, 12:46ï¿½pm, "Lou" <lpog...(a)hotmail.com> wrote:
>
> > > > From the description of the function in the documentation, "TAMARI" should
> > > > encode as T56 - if you're getting anything else, it would appear that you
> > > > have a problem. ï¿½But whether it's a problem with your installation or your
> > > > code, we can't tell. ï¿½It might be helpful if you posted an example of your
> > > > code.
>
> > > > "Nancy" <nancy0...(a)gmail.com> wrote in message
>
> > > >news:c6d64752-83c8-4db6-972f-138850277d73(a)a18g2000yqc.googlegroups.com...
>
> > > > > Hello All,
>
> > > > > I just found a problem when using the soundex function. It singed
> > > > > different values for the same names in my data sets. I am wondering
> > > > > whether there is somehing wrong with my opreation or somthing else.
>
> > > > > Thank you,
>
> > > > > Please see the examples:
>
> > > > > Obs ï¿½ ï¿½First_name ï¿½ ï¿½ ï¿½ ï¿½ IDF
>
> > > > > ï¿½1 ï¿½ ï¿½TAMARI ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ T5623
> > > > > ï¿½2 ï¿½ ï¿½TAMARI ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ T56
> > > > > ï¿½3 ï¿½ ï¿½DEVIN ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½D151
> > > > > ï¿½4 ï¿½ ï¿½DEVIN ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½D15
> > > > > ï¿½5 ï¿½ ï¿½JULIO ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½J42
> > > > > ï¿½6 ï¿½ ï¿½JULIO ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½J4
> > > > > ï¿½7 ï¿½ ï¿½NGOC ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ N221
> > > > > ï¿½8 ï¿½ ï¿½NGOC ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ N22
> > > > > ï¿½9 ï¿½ ï¿½TAMARI ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ T5623
> > > > > 10 ï¿½ ï¿½TAMARI ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ T562- Hide quoted text -
>
> > > > - Show quoted text -- Hide quoted text -
>
> > > - Show quoted text -
>
> > I ran the data you posted and got the same soundex values for each
> > pair. ï¿½So I don't think the problem is SOUNDEX. ï¿½But what could it
> > be? ï¿½Different soundex values imply that some of the words were longer
> > but you see the names as being equal. ï¿½I can think of one way that
> > could happen, I'm sure others can think of otherways. ï¿½Perhaps the
> > NAMES are formatted with a format that does not display the entire
> > value. ï¿½As in this example.
>
> > data test;
> > ï¿½ ï¿½input First_name $16. ï¿½IDF $;
> > ï¿½ ï¿½s = soundex(first_name);
> > ï¿½ ï¿½s2 = soundex(scan(first_name,1,' '));
> > ï¿½ ï¿½format First_name $6.;
> > ï¿½ ï¿½Name = First_name;
> > ï¿½ ï¿½cards;
> > TAMARI J ï¿½ ï¿½ ï¿½ ï¿½ ï¿½T5623
> > TAMARI ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½T56
> > DEVIN ï¿½S ï¿½ ï¿½ ï¿½ ï¿½ ï¿½D151
> > DEVIN ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ D15
> > JULIO ï¿½H ï¿½ ï¿½ ï¿½ ï¿½ ï¿½J42
> > JULIO ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ J4
> > NGOC ï¿½ P ï¿½ ï¿½ ï¿½ ï¿½ ï¿½N221
> > NGOC ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½N22
> > TAMARI ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½T5623
> > TAMARI ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ ï¿½T562
> > ;;;;
> > ï¿½ ï¿½run;
> > proc print;
> > ï¿½ ï¿½run;- Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -
>
> - Show quoted text -

First | Prev |
Pages: 1 2
Prev: what is the main purpose of informat?
Next: Single Hurdle Poisson - MCMC Procedure