bash - comparing two files by lines and removing duplicates from first file -

problem:

1. need compare 2 files,
2. removing duplicate first file
3. then appending lines of file1 file2

illustration example

suppose, 2 files test1 , test2.

$ cat test2 www.xyz.com/abc-2 www.xyz.com/abc-3 www.xyz.com/abc-4 www.xyz.com/abc-5 www.xyz.com/abc-6 

and test1 is

$ cat test1 www.xyz.com/abc-1 www.xyz.com/abc-2 www.xyz.com/abc-3 www.xyz.com/abc-4 www.xyz.com/abc-5 

comparing test1 test2 , removing duplicates test 1

result required:

$ cat test1 www.xyz.com/abc-1 

and adding test1 data in test2

$ cat test2 www.xyz.com/abc-2 www.xyz.com/abc-3 www.xyz.com/abc-4 www.xyz.com/abc-5 www.xyz.com/abc-6 www.xyz.com/abc-1 

solutions tried:

join -v1 -v2 <(sort test1) <(sort test2) 

which resulted (that wrong output)

$ join -v1 -v2 <(sort test1) <(sort test2) www.xyz.com/abc-1 www.xyz.com/abc-6 

another solution tried :

fgrep -vf test1 test2 

which resulted nothing.

with awk:

% awk 'nr == fnr{ a[$0] = 1;next } !a[$0]' test2 test1 www.xyz.com/abc-1 

breakdown:

nr == fnr { # run test2   a[$0] = 1 # store whole line key in associative array   next      # skip next block } !a[$0]      # print line test1 not in