Facing Ambiguous column reference error while inserting into a hive table

Multi tool use
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
My main table:
CREATE EXTERNAL TABLE user(language STRING,snapshot_time STRING,products STRUCT<id:STRING,name:STRING>,item STRUCT<quantity:ARRAY<STRUCT<name:STRING>>>)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS TEXTFILE
LOCATION '/user/input/sample';
This is my main table, from which I'm trying to retrieve specific fields and insert into "user_prod_info" table. But, while inserting data using "Insert into" command, I'm facing the below error :
FAILED: SemanticException [Error 10007]: Ambiguous column reference text in q
CREATE EXTERNAL TABLE user_prod_info (
temp_row_num INT,
language STRING,
snapshot_time STRING,
id STRING,
prod_name STRING,
user_name STRING
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY 't'
NULL DEFINED as "null"
stored as textfile;
My insert command :
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
This command is unable to retrieve the field from the specific table because we have two "name" fields. one is in "products" and the other is in "A".
I tried creating alias for "A.name as name1". I'm able to insert the data without errors. But, one record is storing in 3 rows with some nulls in it.
I got stuck over here. Can anyone please help me out regarding this...
hive
|
show 4 more comments
My main table:
CREATE EXTERNAL TABLE user(language STRING,snapshot_time STRING,products STRUCT<id:STRING,name:STRING>,item STRUCT<quantity:ARRAY<STRUCT<name:STRING>>>)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS TEXTFILE
LOCATION '/user/input/sample';
This is my main table, from which I'm trying to retrieve specific fields and insert into "user_prod_info" table. But, while inserting data using "Insert into" command, I'm facing the below error :
FAILED: SemanticException [Error 10007]: Ambiguous column reference text in q
CREATE EXTERNAL TABLE user_prod_info (
temp_row_num INT,
language STRING,
snapshot_time STRING,
id STRING,
prod_name STRING,
user_name STRING
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY 't'
NULL DEFINED as "null"
stored as textfile;
My insert command :
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
This command is unable to retrieve the field from the specific table because we have two "name" fields. one is in "products" and the other is in "A".
I tried creating alias for "A.name as name1". I'm able to insert the data without errors. But, one record is storing in 3 rows with some nulls in it.
I got stuck over here. Can anyone please help me out regarding this...
hive
It is because your data contains n (newline). Try to put all json object in the single line
– leftjoin
Nov 15 '18 at 15:02
Multi line JSON is not supported: jira.apache.org/jira/browse/HIVE-16829
– leftjoin
Nov 15 '18 at 15:10
I think, the problem is not with the json, because I have removed one of the ambiguous columns and tried to load the data into that table. The data insertion is proper without any nulls.
– fervent
Nov 16 '18 at 5:55
But, when I try to keep the ambiguous column by giving some alias name, then it is inserting one record into 3 rows with nulls in it.
– fervent
Nov 16 '18 at 5:56
1
And question should be renamed to something like Rows splitted when selecting json, this is not an issue with ambiguous column names, you fixed it right
– leftjoin
Nov 16 '18 at 10:34
|
show 4 more comments
My main table:
CREATE EXTERNAL TABLE user(language STRING,snapshot_time STRING,products STRUCT<id:STRING,name:STRING>,item STRUCT<quantity:ARRAY<STRUCT<name:STRING>>>)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS TEXTFILE
LOCATION '/user/input/sample';
This is my main table, from which I'm trying to retrieve specific fields and insert into "user_prod_info" table. But, while inserting data using "Insert into" command, I'm facing the below error :
FAILED: SemanticException [Error 10007]: Ambiguous column reference text in q
CREATE EXTERNAL TABLE user_prod_info (
temp_row_num INT,
language STRING,
snapshot_time STRING,
id STRING,
prod_name STRING,
user_name STRING
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY 't'
NULL DEFINED as "null"
stored as textfile;
My insert command :
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
This command is unable to retrieve the field from the specific table because we have two "name" fields. one is in "products" and the other is in "A".
I tried creating alias for "A.name as name1". I'm able to insert the data without errors. But, one record is storing in 3 rows with some nulls in it.
I got stuck over here. Can anyone please help me out regarding this...
hive
My main table:
CREATE EXTERNAL TABLE user(language STRING,snapshot_time STRING,products STRUCT<id:STRING,name:STRING>,item STRUCT<quantity:ARRAY<STRUCT<name:STRING>>>)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS TEXTFILE
LOCATION '/user/input/sample';
This is my main table, from which I'm trying to retrieve specific fields and insert into "user_prod_info" table. But, while inserting data using "Insert into" command, I'm facing the below error :
FAILED: SemanticException [Error 10007]: Ambiguous column reference text in q
CREATE EXTERNAL TABLE user_prod_info (
temp_row_num INT,
language STRING,
snapshot_time STRING,
id STRING,
prod_name STRING,
user_name STRING
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY 't'
NULL DEFINED as "null"
stored as textfile;
My insert command :
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
This command is unable to retrieve the field from the specific table because we have two "name" fields. one is in "products" and the other is in "A".
I tried creating alias for "A.name as name1". I'm able to insert the data without errors. But, one record is storing in 3 rows with some nulls in it.
I got stuck over here. Can anyone please help me out regarding this...
hive
hive
asked Nov 15 '18 at 12:50


ferventfervent
287
287
It is because your data contains n (newline). Try to put all json object in the single line
– leftjoin
Nov 15 '18 at 15:02
Multi line JSON is not supported: jira.apache.org/jira/browse/HIVE-16829
– leftjoin
Nov 15 '18 at 15:10
I think, the problem is not with the json, because I have removed one of the ambiguous columns and tried to load the data into that table. The data insertion is proper without any nulls.
– fervent
Nov 16 '18 at 5:55
But, when I try to keep the ambiguous column by giving some alias name, then it is inserting one record into 3 rows with nulls in it.
– fervent
Nov 16 '18 at 5:56
1
And question should be renamed to something like Rows splitted when selecting json, this is not an issue with ambiguous column names, you fixed it right
– leftjoin
Nov 16 '18 at 10:34
|
show 4 more comments
It is because your data contains n (newline). Try to put all json object in the single line
– leftjoin
Nov 15 '18 at 15:02
Multi line JSON is not supported: jira.apache.org/jira/browse/HIVE-16829
– leftjoin
Nov 15 '18 at 15:10
I think, the problem is not with the json, because I have removed one of the ambiguous columns and tried to load the data into that table. The data insertion is proper without any nulls.
– fervent
Nov 16 '18 at 5:55
But, when I try to keep the ambiguous column by giving some alias name, then it is inserting one record into 3 rows with nulls in it.
– fervent
Nov 16 '18 at 5:56
1
And question should be renamed to something like Rows splitted when selecting json, this is not an issue with ambiguous column names, you fixed it right
– leftjoin
Nov 16 '18 at 10:34
It is because your data contains n (newline). Try to put all json object in the single line
– leftjoin
Nov 15 '18 at 15:02
It is because your data contains n (newline). Try to put all json object in the single line
– leftjoin
Nov 15 '18 at 15:02
Multi line JSON is not supported: jira.apache.org/jira/browse/HIVE-16829
– leftjoin
Nov 15 '18 at 15:10
Multi line JSON is not supported: jira.apache.org/jira/browse/HIVE-16829
– leftjoin
Nov 15 '18 at 15:10
I think, the problem is not with the json, because I have removed one of the ambiguous columns and tried to load the data into that table. The data insertion is proper without any nulls.
– fervent
Nov 16 '18 at 5:55
I think, the problem is not with the json, because I have removed one of the ambiguous columns and tried to load the data into that table. The data insertion is proper without any nulls.
– fervent
Nov 16 '18 at 5:55
But, when I try to keep the ambiguous column by giving some alias name, then it is inserting one record into 3 rows with nulls in it.
– fervent
Nov 16 '18 at 5:56
But, when I try to keep the ambiguous column by giving some alias name, then it is inserting one record into 3 rows with nulls in it.
– fervent
Nov 16 '18 at 5:56
1
1
And question should be renamed to something like Rows splitted when selecting json, this is not an issue with ambiguous column names, you fixed it right
– leftjoin
Nov 16 '18 at 10:34
And question should be renamed to something like Rows splitted when selecting json, this is not an issue with ambiguous column names, you fixed it right
– leftjoin
Nov 16 '18 at 10:34
|
show 4 more comments
1 Answer
1
active
oldest
votes
you can remove the ambiguous adding an alias for one of the column names like this
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name as prod_name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
I have already added alias name to "A.name as name1". It has inserted properly without any ambiguity. But one record is occupying 3 to 4 rows with nulls in it.
– fervent
Nov 16 '18 at 5:53
@fervent, you should provide input/output data in order to get a better idea
– hlagos
Nov 16 '18 at 13:37
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53319879%2ffacing-ambiguous-column-reference-error-while-inserting-into-a-hive-table%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
you can remove the ambiguous adding an alias for one of the column names like this
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name as prod_name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
I have already added alias name to "A.name as name1". It has inserted properly without any ambiguity. But one record is occupying 3 to 4 rows with nulls in it.
– fervent
Nov 16 '18 at 5:53
@fervent, you should provide input/output data in order to get a better idea
– hlagos
Nov 16 '18 at 13:37
add a comment |
you can remove the ambiguous adding an alias for one of the column names like this
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name as prod_name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
I have already added alias name to "A.name as name1". It has inserted properly without any ambiguity. But one record is occupying 3 to 4 rows with nulls in it.
– fervent
Nov 16 '18 at 5:53
@fervent, you should provide input/output data in order to get a better idea
– hlagos
Nov 16 '18 at 13:37
add a comment |
you can remove the ambiguous adding an alias for one of the column names like this
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name as prod_name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
you can remove the ambiguous adding an alias for one of the column names like this
INSERT OVERWRITE TABLE user_prod_info
SELECT q.* FROM (
SELECT row_number() OVER (PARTITION BY products.id ORDER BY snapshot_time DESC) AS temp_row_num,
language,
snapshot_time,
products.id,
products.name as prod_name,
A.name
FROM user as raw
LATERAL VIEW EXPLODE(item.quantity) quantity as A
) q WHERE temp_row_num == 1;
answered Nov 16 '18 at 1:51
hlagoshlagos
4,5951818
4,5951818
I have already added alias name to "A.name as name1". It has inserted properly without any ambiguity. But one record is occupying 3 to 4 rows with nulls in it.
– fervent
Nov 16 '18 at 5:53
@fervent, you should provide input/output data in order to get a better idea
– hlagos
Nov 16 '18 at 13:37
add a comment |
I have already added alias name to "A.name as name1". It has inserted properly without any ambiguity. But one record is occupying 3 to 4 rows with nulls in it.
– fervent
Nov 16 '18 at 5:53
@fervent, you should provide input/output data in order to get a better idea
– hlagos
Nov 16 '18 at 13:37
I have already added alias name to "A.name as name1". It has inserted properly without any ambiguity. But one record is occupying 3 to 4 rows with nulls in it.
– fervent
Nov 16 '18 at 5:53
I have already added alias name to "A.name as name1". It has inserted properly without any ambiguity. But one record is occupying 3 to 4 rows with nulls in it.
– fervent
Nov 16 '18 at 5:53
@fervent, you should provide input/output data in order to get a better idea
– hlagos
Nov 16 '18 at 13:37
@fervent, you should provide input/output data in order to get a better idea
– hlagos
Nov 16 '18 at 13:37
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53319879%2ffacing-ambiguous-column-reference-error-while-inserting-into-a-hive-table%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
t1ImNBNVh,F 0epquA,xzWafVnMbJIue YKK,8F5JgvVh zM,2fr SEUaAAOIzdjcEN
It is because your data contains n (newline). Try to put all json object in the single line
– leftjoin
Nov 15 '18 at 15:02
Multi line JSON is not supported: jira.apache.org/jira/browse/HIVE-16829
– leftjoin
Nov 15 '18 at 15:10
I think, the problem is not with the json, because I have removed one of the ambiguous columns and tried to load the data into that table. The data insertion is proper without any nulls.
– fervent
Nov 16 '18 at 5:55
But, when I try to keep the ambiguous column by giving some alias name, then it is inserting one record into 3 rows with nulls in it.
– fervent
Nov 16 '18 at 5:56
1
And question should be renamed to something like Rows splitted when selecting json, this is not an issue with ambiguous column names, you fixed it right
– leftjoin
Nov 16 '18 at 10:34