Not receiving document updates on network disconnect and reconnect in Firestore Java admin SDK
I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.
Please help me in debugging this issue.
Edit:
Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:
Scenario 1:
- Watch for query updates
- Client sets up a GRPC connection with the server.
- Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server
- Disconnect the network
- Reconnect within a minute, the query connection is still active and document updates arrive as expected
- Disconnect the network
- Reconnect after 5 minutes
- Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.
- No document updates are received at the client side (as expected)
- Issue a new query and watch for its updates.
- SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.
Scenario 2:
- Watch for query updates
- Disconnect the network for about 5 minutes
- Issue a new query before the network is reconnected.
- SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.
- Reconnect the network.
- Connection attempt of SDK succeeds and the updates start coming for both the queries.
java firebase google-cloud-firestore
add a comment |
I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.
Please help me in debugging this issue.
Edit:
Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:
Scenario 1:
- Watch for query updates
- Client sets up a GRPC connection with the server.
- Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server
- Disconnect the network
- Reconnect within a minute, the query connection is still active and document updates arrive as expected
- Disconnect the network
- Reconnect after 5 minutes
- Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.
- No document updates are received at the client side (as expected)
- Issue a new query and watch for its updates.
- SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.
Scenario 2:
- Watch for query updates
- Disconnect the network for about 5 minutes
- Issue a new query before the network is reconnected.
- SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.
- Reconnect the network.
- Connection attempt of SDK succeeds and the updates start coming for both the queries.
java firebase google-cloud-firestore
add a comment |
I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.
Please help me in debugging this issue.
Edit:
Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:
Scenario 1:
- Watch for query updates
- Client sets up a GRPC connection with the server.
- Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server
- Disconnect the network
- Reconnect within a minute, the query connection is still active and document updates arrive as expected
- Disconnect the network
- Reconnect after 5 minutes
- Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.
- No document updates are received at the client side (as expected)
- Issue a new query and watch for its updates.
- SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.
Scenario 2:
- Watch for query updates
- Disconnect the network for about 5 minutes
- Issue a new query before the network is reconnected.
- SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.
- Reconnect the network.
- Connection attempt of SDK succeeds and the updates start coming for both the queries.
java firebase google-cloud-firestore
I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.
Please help me in debugging this issue.
Edit:
Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:
Scenario 1:
- Watch for query updates
- Client sets up a GRPC connection with the server.
- Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server
- Disconnect the network
- Reconnect within a minute, the query connection is still active and document updates arrive as expected
- Disconnect the network
- Reconnect after 5 minutes
- Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.
- No document updates are received at the client side (as expected)
- Issue a new query and watch for its updates.
- SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.
Scenario 2:
- Watch for query updates
- Disconnect the network for about 5 minutes
- Issue a new query before the network is reconnected.
- SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.
- Reconnect the network.
- Connection attempt of SDK succeeds and the updates start coming for both the queries.
java firebase google-cloud-firestore
java firebase google-cloud-firestore
edited Nov 15 '18 at 14:09
Sudarshan Murthy
asked Nov 15 '18 at 6:44
Sudarshan MurthySudarshan Murthy
7711
7711
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.
The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.
As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.
Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.
– Sudarshan Murthy
Nov 15 '18 at 14:13
If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.
– Alex Mamo
Nov 15 '18 at 14:16
Thank you. I have raised an issue in the github tracker
– Sudarshan Murthy
Nov 15 '18 at 14:57
Ok, please keep me also posted. Thanks!
– Alex Mamo
Nov 15 '18 at 15:53
Looks like there is a setting to enable the GRPC keepalive in theFirestoreOptions
while initializing theFirebaseApp
. Please see my answer for details
– Sudarshan Murthy
Nov 19 '18 at 5:58
add a comment |
The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
This can be done by providing TransportChannelProvider
in the FirestoreOptions
while initializing the FirebaseApp
.
The code snippet is given below:
InstantiatingGrpcChannelProvider channelProvider =
InstantiatingGrpcChannelProvider.newBuilder()
.setKeepAliveTime(Duration.ofSeconds(60L))
.setKeepAliveTimeout(Duration.ofMinutes(5L))
.build();
FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
.setChannelProvider(channelProvider).build();
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(credentials).setFirestoreOptions(firestoreOptions)
.setConnectTimeout(5000).setReadTimeout(5000).build();
FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);
Firestore firestore = FirestoreClient.getFirestore(firebaseApp);
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53313824%2fnot-receiving-document-updates-on-network-disconnect-and-reconnect-in-firestore%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.
The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.
As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.
Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.
– Sudarshan Murthy
Nov 15 '18 at 14:13
If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.
– Alex Mamo
Nov 15 '18 at 14:16
Thank you. I have raised an issue in the github tracker
– Sudarshan Murthy
Nov 15 '18 at 14:57
Ok, please keep me also posted. Thanks!
– Alex Mamo
Nov 15 '18 at 15:53
Looks like there is a setting to enable the GRPC keepalive in theFirestoreOptions
while initializing theFirebaseApp
. Please see my answer for details
– Sudarshan Murthy
Nov 19 '18 at 5:58
add a comment |
When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.
The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.
As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.
Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.
– Sudarshan Murthy
Nov 15 '18 at 14:13
If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.
– Alex Mamo
Nov 15 '18 at 14:16
Thank you. I have raised an issue in the github tracker
– Sudarshan Murthy
Nov 15 '18 at 14:57
Ok, please keep me also posted. Thanks!
– Alex Mamo
Nov 15 '18 at 15:53
Looks like there is a setting to enable the GRPC keepalive in theFirestoreOptions
while initializing theFirebaseApp
. Please see my answer for details
– Sudarshan Murthy
Nov 19 '18 at 5:58
add a comment |
When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.
The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.
As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.
When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.
The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.
As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.
answered Nov 15 '18 at 9:33
Alex MamoAlex Mamo
47k82965
47k82965
Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.
– Sudarshan Murthy
Nov 15 '18 at 14:13
If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.
– Alex Mamo
Nov 15 '18 at 14:16
Thank you. I have raised an issue in the github tracker
– Sudarshan Murthy
Nov 15 '18 at 14:57
Ok, please keep me also posted. Thanks!
– Alex Mamo
Nov 15 '18 at 15:53
Looks like there is a setting to enable the GRPC keepalive in theFirestoreOptions
while initializing theFirebaseApp
. Please see my answer for details
– Sudarshan Murthy
Nov 19 '18 at 5:58
add a comment |
Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.
– Sudarshan Murthy
Nov 15 '18 at 14:13
If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.
– Alex Mamo
Nov 15 '18 at 14:16
Thank you. I have raised an issue in the github tracker
– Sudarshan Murthy
Nov 15 '18 at 14:57
Ok, please keep me also posted. Thanks!
– Alex Mamo
Nov 15 '18 at 15:53
Looks like there is a setting to enable the GRPC keepalive in theFirestoreOptions
while initializing theFirebaseApp
. Please see my answer for details
– Sudarshan Murthy
Nov 19 '18 at 5:58
Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.
– Sudarshan Murthy
Nov 15 '18 at 14:13
Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.
– Sudarshan Murthy
Nov 15 '18 at 14:13
If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.
– Alex Mamo
Nov 15 '18 at 14:16
If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.
– Alex Mamo
Nov 15 '18 at 14:16
Thank you. I have raised an issue in the github tracker
– Sudarshan Murthy
Nov 15 '18 at 14:57
Thank you. I have raised an issue in the github tracker
– Sudarshan Murthy
Nov 15 '18 at 14:57
Ok, please keep me also posted. Thanks!
– Alex Mamo
Nov 15 '18 at 15:53
Ok, please keep me also posted. Thanks!
– Alex Mamo
Nov 15 '18 at 15:53
Looks like there is a setting to enable the GRPC keepalive in the
FirestoreOptions
while initializing the FirebaseApp
. Please see my answer for details– Sudarshan Murthy
Nov 19 '18 at 5:58
Looks like there is a setting to enable the GRPC keepalive in the
FirestoreOptions
while initializing the FirebaseApp
. Please see my answer for details– Sudarshan Murthy
Nov 19 '18 at 5:58
add a comment |
The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
This can be done by providing TransportChannelProvider
in the FirestoreOptions
while initializing the FirebaseApp
.
The code snippet is given below:
InstantiatingGrpcChannelProvider channelProvider =
InstantiatingGrpcChannelProvider.newBuilder()
.setKeepAliveTime(Duration.ofSeconds(60L))
.setKeepAliveTimeout(Duration.ofMinutes(5L))
.build();
FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
.setChannelProvider(channelProvider).build();
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(credentials).setFirestoreOptions(firestoreOptions)
.setConnectTimeout(5000).setReadTimeout(5000).build();
FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);
Firestore firestore = FirestoreClient.getFirestore(firebaseApp);
add a comment |
The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
This can be done by providing TransportChannelProvider
in the FirestoreOptions
while initializing the FirebaseApp
.
The code snippet is given below:
InstantiatingGrpcChannelProvider channelProvider =
InstantiatingGrpcChannelProvider.newBuilder()
.setKeepAliveTime(Duration.ofSeconds(60L))
.setKeepAliveTimeout(Duration.ofMinutes(5L))
.build();
FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
.setChannelProvider(channelProvider).build();
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(credentials).setFirestoreOptions(firestoreOptions)
.setConnectTimeout(5000).setReadTimeout(5000).build();
FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);
Firestore firestore = FirestoreClient.getFirestore(firebaseApp);
add a comment |
The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
This can be done by providing TransportChannelProvider
in the FirestoreOptions
while initializing the FirebaseApp
.
The code snippet is given below:
InstantiatingGrpcChannelProvider channelProvider =
InstantiatingGrpcChannelProvider.newBuilder()
.setKeepAliveTime(Duration.ofSeconds(60L))
.setKeepAliveTimeout(Duration.ofMinutes(5L))
.build();
FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
.setChannelProvider(channelProvider).build();
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(credentials).setFirestoreOptions(firestoreOptions)
.setConnectTimeout(5000).setReadTimeout(5000).build();
FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);
Firestore firestore = FirestoreClient.getFirestore(firebaseApp);
The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
This can be done by providing TransportChannelProvider
in the FirestoreOptions
while initializing the FirebaseApp
.
The code snippet is given below:
InstantiatingGrpcChannelProvider channelProvider =
InstantiatingGrpcChannelProvider.newBuilder()
.setKeepAliveTime(Duration.ofSeconds(60L))
.setKeepAliveTimeout(Duration.ofMinutes(5L))
.build();
FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
.setChannelProvider(channelProvider).build();
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(credentials).setFirestoreOptions(firestoreOptions)
.setConnectTimeout(5000).setReadTimeout(5000).build();
FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);
Firestore firestore = FirestoreClient.getFirestore(firebaseApp);
answered Nov 19 '18 at 5:56
Sudarshan MurthySudarshan Murthy
7711
7711
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53313824%2fnot-receiving-document-updates-on-network-disconnect-and-reconnect-in-firestore%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown