Pfthb

1.streaming data is coming from kafka
2.consuming through spark streaming
3.firstname,lastname,userid and membername ( using member names i am getting the member count
e.g mark,tyson,2,chris,lisa,iwanka - so here member count is 3

somehow i have to do the count its the requirmnt . but how can i remove deduplication after aggregation . its my concern

 val df2=df.select(firstname,lastname,membercount,userid)
 df2.writestream.format("console").start().awaitTermination

 or 
 df3.select("*").where("membercount >= 3").dropDuplication("userid")

 // this one is not working , but i need to do the same after
 count only so that in batches same user id will not come again.
 only first time entry i want.

Batch-1 output

 firstname lastname member-count userid

 john smith 5 1
 mark boucher 8 2
 shawn pollock 3 3

batch-2 output

 firstname lastname member-count userid

 john smith 7 (prev.count 5) 1
shawn pollock 12 (prev.count 8) 3
chris jordan 6 4

// but here i want batch -2 ---------output

1.The possibilty is the john smith ,shawn pollock count will increase again in next batches ,but i dont want to show or keep in output for next batches.

i.e based on userid , i want entry for the one time only in batch output
and neglect again the same user in batch output
firstname lastname member-count userid
chris jordan 6 4

Your question is hard to read, but as I understand you want a while loop with a condition?

var a = 10;
while(a < 20)
 println( "Value of a: " + a );
 a = a + 1;
 

For example will print

value of a: 10
value of a: 11
value of a: 12
value of a: 13
value of a: 14
value of a: 15
value of a: 16
value of a: 17
value of a: 18
value of a: 19