$expr arrayElementAt not working in aggregation for embedded document
I am doing mongo db aggregation like
$cursor = $this->collection->aggregate(
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT'
)
),
array(
'$match' =>
array(
'$or' => array(
array("FullName" => new MongoDBBSONRegex($arg, 'i')),
array("FirstLast" => new MongoDBBSONRegex($arg, 'i')),
array("FirstMiddle" => new MongoDBBSONRegex($arg, 'i')),
array("FirstName" => new MongoDBBSONRegex($arg, 'i')),
array("MiddleName" => new MongoDBBSONRegex($arg, 'i')),
array("LastName" => new MongoDBBSONRegex($arg, 'i')),
array("Student.registration_temp_perm_no" => $arg),
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)),
),
// "Student.assigned_keys" => ['$exists' => false],
"Student.schoolId" => new MongoDBBSONObjectID($this->SchoolId)
)
)
)
);
I have collection which contains data like
"first_name": "John",
"middle_name": "",
"last_name": "Mayor",
"allotment_details": [
"allotment_id": "ff666d55-2fcc-79b2-e4da-e165939555bb",
"room_id": "5be2d9aad2ccda0fdc006a65",
"bay_id": ObjectId("5be2d9aad2ccda0fdc006a61"),
...
The above code is working for fine concatenation of three names types and searching all the data passed in $arg
. Note that I have added array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId))
in order to fetch students based on concatenation of names and those students should be fetched on the basis of $this->RoomId
.
The above code is not fetching students assigned to a room but is fetching all the students based on concatenation of names.
Please help !!!
mongodb mongodb-query aggregation-framework mongodb-php php-mongodb
add a comment |
I am doing mongo db aggregation like
$cursor = $this->collection->aggregate(
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT'
)
),
array(
'$match' =>
array(
'$or' => array(
array("FullName" => new MongoDBBSONRegex($arg, 'i')),
array("FirstLast" => new MongoDBBSONRegex($arg, 'i')),
array("FirstMiddle" => new MongoDBBSONRegex($arg, 'i')),
array("FirstName" => new MongoDBBSONRegex($arg, 'i')),
array("MiddleName" => new MongoDBBSONRegex($arg, 'i')),
array("LastName" => new MongoDBBSONRegex($arg, 'i')),
array("Student.registration_temp_perm_no" => $arg),
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)),
),
// "Student.assigned_keys" => ['$exists' => false],
"Student.schoolId" => new MongoDBBSONObjectID($this->SchoolId)
)
)
)
);
I have collection which contains data like
"first_name": "John",
"middle_name": "",
"last_name": "Mayor",
"allotment_details": [
"allotment_id": "ff666d55-2fcc-79b2-e4da-e165939555bb",
"room_id": "5be2d9aad2ccda0fdc006a65",
"bay_id": ObjectId("5be2d9aad2ccda0fdc006a61"),
...
The above code is working for fine concatenation of three names types and searching all the data passed in $arg
. Note that I have added array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId))
in order to fetch students based on concatenation of names and those students should be fetched on the basis of $this->RoomId
.
The above code is not fetching students assigned to a room but is fetching all the students based on concatenation of names.
Please help !!!
mongodb mongodb-query aggregation-framework mongodb-php php-mongodb
1
allotment_details
is not included in your initial$project
, so the field is not there.$project
and$group
only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
– Neil Lunn
Nov 13 '18 at 10:04
1
At any rate$match
should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using$or
instead of concatenating strings.
– Neil Lunn
Nov 13 '18 at 10:06
can you please provide code ....
– Nisa Nisa
Nov 13 '18 at 10:08
add a comment |
I am doing mongo db aggregation like
$cursor = $this->collection->aggregate(
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT'
)
),
array(
'$match' =>
array(
'$or' => array(
array("FullName" => new MongoDBBSONRegex($arg, 'i')),
array("FirstLast" => new MongoDBBSONRegex($arg, 'i')),
array("FirstMiddle" => new MongoDBBSONRegex($arg, 'i')),
array("FirstName" => new MongoDBBSONRegex($arg, 'i')),
array("MiddleName" => new MongoDBBSONRegex($arg, 'i')),
array("LastName" => new MongoDBBSONRegex($arg, 'i')),
array("Student.registration_temp_perm_no" => $arg),
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)),
),
// "Student.assigned_keys" => ['$exists' => false],
"Student.schoolId" => new MongoDBBSONObjectID($this->SchoolId)
)
)
)
);
I have collection which contains data like
"first_name": "John",
"middle_name": "",
"last_name": "Mayor",
"allotment_details": [
"allotment_id": "ff666d55-2fcc-79b2-e4da-e165939555bb",
"room_id": "5be2d9aad2ccda0fdc006a65",
"bay_id": ObjectId("5be2d9aad2ccda0fdc006a61"),
...
The above code is working for fine concatenation of three names types and searching all the data passed in $arg
. Note that I have added array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId))
in order to fetch students based on concatenation of names and those students should be fetched on the basis of $this->RoomId
.
The above code is not fetching students assigned to a room but is fetching all the students based on concatenation of names.
Please help !!!
mongodb mongodb-query aggregation-framework mongodb-php php-mongodb
I am doing mongo db aggregation like
$cursor = $this->collection->aggregate(
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT'
)
),
array(
'$match' =>
array(
'$or' => array(
array("FullName" => new MongoDBBSONRegex($arg, 'i')),
array("FirstLast" => new MongoDBBSONRegex($arg, 'i')),
array("FirstMiddle" => new MongoDBBSONRegex($arg, 'i')),
array("FirstName" => new MongoDBBSONRegex($arg, 'i')),
array("MiddleName" => new MongoDBBSONRegex($arg, 'i')),
array("LastName" => new MongoDBBSONRegex($arg, 'i')),
array("Student.registration_temp_perm_no" => $arg),
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)),
),
// "Student.assigned_keys" => ['$exists' => false],
"Student.schoolId" => new MongoDBBSONObjectID($this->SchoolId)
)
)
)
);
I have collection which contains data like
"first_name": "John",
"middle_name": "",
"last_name": "Mayor",
"allotment_details": [
"allotment_id": "ff666d55-2fcc-79b2-e4da-e165939555bb",
"room_id": "5be2d9aad2ccda0fdc006a65",
"bay_id": ObjectId("5be2d9aad2ccda0fdc006a61"),
...
The above code is working for fine concatenation of three names types and searching all the data passed in $arg
. Note that I have added array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId))
in order to fetch students based on concatenation of names and those students should be fetched on the basis of $this->RoomId
.
The above code is not fetching students assigned to a room but is fetching all the students based on concatenation of names.
Please help !!!
mongodb mongodb-query aggregation-framework mongodb-php php-mongodb
mongodb mongodb-query aggregation-framework mongodb-php php-mongodb
edited Nov 13 '18 at 10:12
Raghbendra Nayak
1,16521232
1,16521232
asked Nov 13 '18 at 9:56
Nisa NisaNisa Nisa
186
186
1
allotment_details
is not included in your initial$project
, so the field is not there.$project
and$group
only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
– Neil Lunn
Nov 13 '18 at 10:04
1
At any rate$match
should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using$or
instead of concatenating strings.
– Neil Lunn
Nov 13 '18 at 10:06
can you please provide code ....
– Nisa Nisa
Nov 13 '18 at 10:08
add a comment |
1
allotment_details
is not included in your initial$project
, so the field is not there.$project
and$group
only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
– Neil Lunn
Nov 13 '18 at 10:04
1
At any rate$match
should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using$or
instead of concatenating strings.
– Neil Lunn
Nov 13 '18 at 10:06
can you please provide code ....
– Nisa Nisa
Nov 13 '18 at 10:08
1
1
allotment_details
is not included in your initial $project
, so the field is not there. $project
and $group
only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".– Neil Lunn
Nov 13 '18 at 10:04
allotment_details
is not included in your initial $project
, so the field is not there. $project
and $group
only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".– Neil Lunn
Nov 13 '18 at 10:04
1
1
At any rate
$match
should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using $or
instead of concatenating strings.– Neil Lunn
Nov 13 '18 at 10:06
At any rate
$match
should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using $or
instead of concatenating strings.– Neil Lunn
Nov 13 '18 at 10:06
can you please provide code ....
– Nisa Nisa
Nov 13 '18 at 10:08
can you please provide code ....
– Nisa Nisa
Nov 13 '18 at 10:08
add a comment |
1 Answer
1
active
oldest
votes
Quick fix
Your "pipeline" does not work here primarily because your initial $project
lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),
Or even since you used $$ROOT
for Student
anyway, simply qualify the field under that path:
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),
however I would strongly* implore that you do NOT do that.
The whole concept of "concatenating strings" in order to do a later $match
on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.
Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position
or possibly even the $sort
modifiers to $push
do for you, by changing where items get added or the sorted order of items respectively.
Changing the Array to "newest first"
This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr
argument.
The core concepts are to "pre-pend" new array items with syntax like:
$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)
Where $alloments
must be an array as required by $each
and $position
is used to 0
in order to add the new array item "first".
Alternately if you actually have something like created_date
as a property within each of the objects in the array, then you "could" use something like $sort
as a modifier instead.
$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)
It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date
or other "sort" property in a way which would effect the order of the array elements when "sorted".
The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:
$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])
MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0
index. What you cannot do is specify a "negative" index i.e:
$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])
That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.
Concatenation is Bad
The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or
with the conditions on each of the fields as they exist already within the actual document:
$this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.
Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:
$this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat
"brute force" approach simply cannot deliver.
N.B Modern PHP Releases generally support
as a much more brief representation of
array()
. It's a lot cleaner and far easier to read. So please use it.
Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 '18 at 12:25
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53278310%2fexpr-arrayelementat-not-working-in-aggregation-for-embedded-document%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Quick fix
Your "pipeline" does not work here primarily because your initial $project
lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),
Or even since you used $$ROOT
for Student
anyway, simply qualify the field under that path:
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),
however I would strongly* implore that you do NOT do that.
The whole concept of "concatenating strings" in order to do a later $match
on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.
Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position
or possibly even the $sort
modifiers to $push
do for you, by changing where items get added or the sorted order of items respectively.
Changing the Array to "newest first"
This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr
argument.
The core concepts are to "pre-pend" new array items with syntax like:
$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)
Where $alloments
must be an array as required by $each
and $position
is used to 0
in order to add the new array item "first".
Alternately if you actually have something like created_date
as a property within each of the objects in the array, then you "could" use something like $sort
as a modifier instead.
$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)
It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date
or other "sort" property in a way which would effect the order of the array elements when "sorted".
The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:
$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])
MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0
index. What you cannot do is specify a "negative" index i.e:
$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])
That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.
Concatenation is Bad
The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or
with the conditions on each of the fields as they exist already within the actual document:
$this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.
Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:
$this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat
"brute force" approach simply cannot deliver.
N.B Modern PHP Releases generally support
as a much more brief representation of
array()
. It's a lot cleaner and far easier to read. So please use it.
Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 '18 at 12:25
add a comment |
Quick fix
Your "pipeline" does not work here primarily because your initial $project
lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),
Or even since you used $$ROOT
for Student
anyway, simply qualify the field under that path:
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),
however I would strongly* implore that you do NOT do that.
The whole concept of "concatenating strings" in order to do a later $match
on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.
Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position
or possibly even the $sort
modifiers to $push
do for you, by changing where items get added or the sorted order of items respectively.
Changing the Array to "newest first"
This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr
argument.
The core concepts are to "pre-pend" new array items with syntax like:
$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)
Where $alloments
must be an array as required by $each
and $position
is used to 0
in order to add the new array item "first".
Alternately if you actually have something like created_date
as a property within each of the objects in the array, then you "could" use something like $sort
as a modifier instead.
$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)
It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date
or other "sort" property in a way which would effect the order of the array elements when "sorted".
The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:
$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])
MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0
index. What you cannot do is specify a "negative" index i.e:
$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])
That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.
Concatenation is Bad
The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or
with the conditions on each of the fields as they exist already within the actual document:
$this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.
Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:
$this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat
"brute force" approach simply cannot deliver.
N.B Modern PHP Releases generally support
as a much more brief representation of
array()
. It's a lot cleaner and far easier to read. So please use it.
Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 '18 at 12:25
add a comment |
Quick fix
Your "pipeline" does not work here primarily because your initial $project
lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),
Or even since you used $$ROOT
for Student
anyway, simply qualify the field under that path:
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),
however I would strongly* implore that you do NOT do that.
The whole concept of "concatenating strings" in order to do a later $match
on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.
Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position
or possibly even the $sort
modifiers to $push
do for you, by changing where items get added or the sorted order of items respectively.
Changing the Array to "newest first"
This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr
argument.
The core concepts are to "pre-pend" new array items with syntax like:
$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)
Where $alloments
must be an array as required by $each
and $position
is used to 0
in order to add the new array item "first".
Alternately if you actually have something like created_date
as a property within each of the objects in the array, then you "could" use something like $sort
as a modifier instead.
$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)
It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date
or other "sort" property in a way which would effect the order of the array elements when "sorted".
The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:
$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])
MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0
index. What you cannot do is specify a "negative" index i.e:
$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])
That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.
Concatenation is Bad
The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or
with the conditions on each of the fields as they exist already within the actual document:
$this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.
Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:
$this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat
"brute force" approach simply cannot deliver.
N.B Modern PHP Releases generally support
as a much more brief representation of
array()
. It's a lot cleaner and far easier to read. So please use it.
Quick fix
Your "pipeline" does not work here primarily because your initial $project
lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),
Or even since you used $$ROOT
for Student
anyway, simply qualify the field under that path:
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),
however I would strongly* implore that you do NOT do that.
The whole concept of "concatenating strings" in order to do a later $match
on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.
Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position
or possibly even the $sort
modifiers to $push
do for you, by changing where items get added or the sorted order of items respectively.
Changing the Array to "newest first"
This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr
argument.
The core concepts are to "pre-pend" new array items with syntax like:
$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)
Where $alloments
must be an array as required by $each
and $position
is used to 0
in order to add the new array item "first".
Alternately if you actually have something like created_date
as a property within each of the objects in the array, then you "could" use something like $sort
as a modifier instead.
$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)
It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date
or other "sort" property in a way which would effect the order of the array elements when "sorted".
The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:
$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])
MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0
index. What you cannot do is specify a "negative" index i.e:
$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])
That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.
Concatenation is Bad
The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or
with the conditions on each of the fields as they exist already within the actual document:
$this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.
Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:
$this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])
Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat
"brute force" approach simply cannot deliver.
N.B Modern PHP Releases generally support
as a much more brief representation of
array()
. It's a lot cleaner and far easier to read. So please use it.
edited Nov 14 '18 at 2:53
answered Nov 13 '18 at 10:55
Neil LunnNeil Lunn
98.1k23174184
98.1k23174184
Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 '18 at 12:25
add a comment |
Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 '18 at 12:25
Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 '18 at 12:25
Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 '18 at 12:25
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53278310%2fexpr-arrayelementat-not-working-in-aggregation-for-embedded-document%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
allotment_details
is not included in your initial$project
, so the field is not there.$project
and$group
only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".– Neil Lunn
Nov 13 '18 at 10:04
1
At any rate
$match
should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using$or
instead of concatenating strings.– Neil Lunn
Nov 13 '18 at 10:06
can you please provide code ....
– Nisa Nisa
Nov 13 '18 at 10:08