Data Mining

Download Free Data Mining Source Code In C/C++, C#, Visual Basic, Visual Basic.NET, Java,
and other programming languages
Welcome to Data Mining Sign in | Join | Help
in Search

Data Mining Source Code Newsletter

Business Analyst Training
Live, Online, Video Courses
Instructor-Led + Hands-On
BusinessAnalystBootCamp.Com

SQL + Database Training
Live, Online, Video Classes
Instructor-Led + Hands-On
SQLBootCamp.Com

Software Developer Training
Live, Online, Video Courses
Instructor-Led + Hands-On
SoftwareDevelperBootCamp.Com

IT CAREER COACH
Hands-On Experience Coaching
IT Skills Training
IT-Career-Coach.NET

IT Professional Newsletter
"Free" IT Career Success Tips
How To Accelerate Your Career
IT Career Newsletter

Ask IT Career Questions
"ASK" A Burning IT Career
Question Or Get Answers
Ask A Burning IT Question Now!

Announcing The Data Mining Source Code Newsletter!

Subscribe By Email | Subscribe By RSS Feed

detect and adding missing rows repetively

Last post 07-27-2009, 8:30 by hunterdong. 9 replies.
Sort Posts: Previous Next
  •  06-16-2009, 3:19 8929

    detect and adding missing rows repetively

    Hi,

     I have got one table which contains sales records for each week:

    Week  ProductName Value

    wk1    ProdA    5

    wk1   ProdB    3

    wk2   ProdA   4

    wk2  ProdC    5

     

    If a certain product is not sold that week, it is not stored as a row for that week.

    If I want to insert these non-sold product with 0 value, will I be able to do this?

     I can do this if the table only contains one week, using a full week and anti-join with the week and then append.

    Can anyone think about a way of doing it for multiple weeks? Thanks.

  •  06-18-2009, 3:52 8934 in reply to 8929

    Re: detect and adding missing rows repetively

    Hi,

    I think that you should create a table with all weeks (1 to 52) and all products (ProdA to ProdZ) (=> 2 fields)
    filter Value's field from sales's file
    merge (anti join, first file with new table)
    merge (full outer join) merge + sales's file
    filler (based on condition: @NULL(Value) => 0)
    field reorder (Week / ProductName / Value)
    sort (Week / ProductName)

    you should use TRIM() function before to merge the fields because the sales's file are not "clean" (space)

     

    Filed under:
  •  06-19-2009, 5:37 8935 in reply to 8934

    Re: detect and adding missing rows repetively

    Thank you very much DavidF, I think that is the solution .

    Can I ask if it is some pattern you created yourself, or is there any tutorial or help file mentioned this method? It is quite useful so I want to find out more about it.

    My summary of your full outer join is: out join can be used as append if there is no intersection
     

     

     

     

  •  06-19-2009, 6:35 8936 in reply to 8935

    Re: detect and adding missing rows repetively

    No tutorial, it was created just for you.

    and for me to do exercise of course.

     

     

     

  •  06-19-2009, 6:36 8937 in reply to 8935

    Re: detect and adding missing rows repetively

    The method works if the artificial table created contains exactly the same weeks as the actual sales file.

     Which can be implemented easily by using Tim's cartisian method (one product list and another aggregation for week/year combination.

     

  •  06-19-2009, 8:39 8938 in reply to 8937

    Re: detect and adding missing rows repetively

    One alteration:

     Not out-join

    but Append, otherwise you receive "duplicated fields" error.

    my previous comment of "outer join has same effect as append" is wrong. 

     

  •  06-20-2009, 2:20 9025 in reply to 8938

    Re: detect and adding missing rows repetively

    "The method works if the artificial table created contains exactly the same weeks as the actual sales file"

    of course, otherwise it's impossible to merge. that's why you shoud use the trim() function.

     if you don't know exactly the name of the future weeks, then just create an artificial table until the last week you use.

    before to post, I used your data set and received no problem with merge / full outer join.

    so, tell me the way to post my stream and I'll do it...

     

  •  06-22-2009, 5:38 9027 in reply to 9025

    Re: detect and adding missing rows repetively

    HI DavidF,

     

    it is "options" tab in replying, there you can attach a file.

     I made it work with Append node, I don't understanding how you can merge missing records with non-missing records and make it a complete list.

  •  07-14-2009, 1:30 9084 in reply to 9027

    Re: detect and adding missing rows repetively

    haha, I think I know what DavidF means.

     Use one partial outer join only, the artificial table has only the key fields, so it only brings in the missing keys, other fields will be automatically created.

     

  •  07-27-2009, 8:30 9111 in reply to 9084

    Re: detect and adding missing rows repetively

    Found a simpler way:

     Create a complete list of every week and every product, then partial outer join with existing records.......

Announcing The Data Mining Source Code Newsletter!

Subscribe By Email | Subscribe By RSS Feed