Remove duplicates

[c740015]

Hi im Trying to remove duplicate entries from a text file i load it will grab a string and display the array
also i asked it to save the array to a textfile

Problem is the text file has duplicates in and i cant get rid of them no matter what i do..

Im really stuck at this part i would love some help..

Here is my code

<?
$loadfile = file_get_contents("codes.txt");
 
$expression = "#[A-Z0-9]{8}([-_][A-Z0-9]{8}){3}#";
$resultat = Array("#[A-Z0-9]{8}([-_][A-Z0-9]{8}){3}#");
$file="test.txt";
 
preg_match_all($expression, $loadfile, $resultat, PREG_PATTERN_ORDER);

echo "<pre>";
   print_r(array_unique($resultat));
echo "</pre>";
 
ob_start();
print_r(array_unique($resultat));
$result = ob_get_contents();
ob_end_clean();

$handle=fopen($file, "w"); 
fwrite($handle, $result);
fclose($handle);
 
?>

This is the out put of the code

Array
(
[0] => Array
(
[0] => 00000000-00000000-00000000-00000001
[1] => 00000000-00000000-00000000-0000000B
[2] => 00000000-00000000-00000000-0000000F
[3] => 00000000-00000000-00000000-00000016
[4] => 00000000-00000000-00000000-00010001
[74] => 00000000-00000000-00000000-00000001
[75] => 00000000-00000000-00000000-0000000B
[76] => 00000000-00000000-00000000-0000000F
[77] => 00000000-00000000-00000000-00000001
[78] => 00000000-00000000-00000000-0000000B
[79] => 00000000-00000000-00000000-0000000F
[80] => 00000000-00000000-00000000-00000016

As you can see its duplicating the value.
either to remove the key and kill duplicates that way or a method to just kill the duplicates..

Im really stuck

Any help would be amazing.

[misson]

The issue with your code is that $resultat is a multidimensional array with a single value. The array_unique is called on the wrong value; a $resultat = $resultat[0]; would fix this, but the performance can be improved by taking a different approach.

PHP uses associative arrays. Store the lines as keys rather than values:

function getCodes($fname) {
    $codes = array();
    $codeFile = fopen($fname, 'r');
    if ($codeFile) {
        while (FALSE !== ($line = fgets($codeFile))) {
            preg_match('/[A-Z0-9]{8}([-_][A-Z0-9]{8}){3}/', $line, $match);
            if ($match) {
                $codes[$match[0]] = $match[0];
            }
        }
        fclose($codeFile);
    }
    return $codes;
} 

You might want to set your own error handler. I've switched from file_get_contents &c to the wrapped C file I/O functions to reduce memory usage, but if the files are small enough, you don't need to make this change.

Note that you could write this as a very simple shell script. If the file lines only contain the codes, all you'd need is

#!/bin/bash
sort $1 | uniq > $2

[c740015]

Nice Im gonna get on this now cheers ..

Im a little confused but i think i understand ur reply im new to php Im just learning..

I do have some programming knowledge but not so much in php.

Ill let u know how i get on
Edit:
Ugh.. Im still stuck

I did try the 1st option $resultat = $resultat[0];

Unfortunatly still displayig exactly the same information loads of duplicates

And the Function i tried to use that also..

and im getting this error Warning: fclose(): supplied argument is not a valid stream resource in

[slacker3]

does it have to be done with PHP ?

otherwise you should really use sort/uniq, since it does exactly what you want with just one line of code

if you don't have a linux box:
http://www.cygwin.com/

[c740015]

Yeah cgwin is no good for me.. I want users to input the codes into a text box then i can parse them and then remove the duplicates...

Having linux installed on my machine is no good..

and since the user can be puting anything in the box it really does need to grab the 32 digits.

The only problem i am having is removing duplicates..



Once that is done im all set.

Bascially i have set up a text box that people drop there codes in then i just run this script to parse them.

But i really need duplicates to be removed or it would be no use at all.

I have spent alot of time on it reading various sites and im totally puzzled and cant think what to do now

[slacker3]

1. Use file() function to read a file. this will give the file data in array
2. Now use array_unique function. This will remove duplicate enntries.
3. Now write the contents of the array in to file.

http://forums.digitalpoint.com/showthread.php?t=288961

http://www.ozzu.com/programming-foru...hp-t68080.html
http://www.jonasjohn.de/snippets/php...ated-lines.htm

[misson]

I did try the 1st option $resultat = $resultat[0];

Unfortunatly still displayig exactly the same information loads of duplicates

Where did you place the statement? After the call to preg_match_all? The following works for me

$loadfile = file_get_contents("codes.txt");
$expression = "#[A-Z0-9]{8}([-_][A-Z0-9]{8}){3}#";
preg_match_all($expression, $loadfile, $resultat, PREG_PATTERN_ORDER);
$resultat = array_unique($resultat[0]); 

array_unique is likely to be the most expensive (in terms of time) part of the script, so don't call it more than you must. Better not to use it at all.

And the Function i tried to use that also..

and im getting this error Warning: fclose(): supplied argument is not a valid stream resource in

It's been fixed in the code; try it again.

[c740015]

Nice one sorry for the late reply..

Mission great guy thanks very much for your help worked perfectly...

Any idea how i would insert each code into a database and pull them out again to show on a different page so i can store a database record of each code.. My goal is to have another screen where i can sort each code into catagories or some thing like that or just to store them in a databse..

I wanna learn more..

Thanks mission rep for that...

[misson]

Any idea how i would insert each code into a database and pull them out again

Do you mean you want to learn SQL? Try SQLzoo and "Writing MySQL Scripts with PHP and PDO", or the suggestions in
other threads ("sql database", "Sql for noobs", "Want to learn to program...."). The threads include links to yet more pages with yet more suggestions.