Not receiving document updates on network disconnect and reconnect in Firestore Java admin SDK










0















I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.



Please help me in debugging this issue.



Edit:



Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:




  • Scenario 1:



    1. Watch for query updates

    2. Client sets up a GRPC connection with the server.

    3. Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server

    4. Disconnect the network

    5. Reconnect within a minute, the query connection is still active and document updates arrive as expected

    6. Disconnect the network

    7. Reconnect after 5 minutes

    8. Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.

    9. No document updates are received at the client side (as expected)

    10. Issue a new query and watch for its updates.

    11. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.



  • Scenario 2:



    1. Watch for query updates

    2. Disconnect the network for about 5 minutes

    3. Issue a new query before the network is reconnected.

    4. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.

    5. Reconnect the network.

    6. Connection attempt of SDK succeeds and the updates start coming for both the queries.










share|improve this question




























    0















    I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.



    Please help me in debugging this issue.



    Edit:



    Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:




    • Scenario 1:



      1. Watch for query updates

      2. Client sets up a GRPC connection with the server.

      3. Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server

      4. Disconnect the network

      5. Reconnect within a minute, the query connection is still active and document updates arrive as expected

      6. Disconnect the network

      7. Reconnect after 5 minutes

      8. Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.

      9. No document updates are received at the client side (as expected)

      10. Issue a new query and watch for its updates.

      11. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.



    • Scenario 2:



      1. Watch for query updates

      2. Disconnect the network for about 5 minutes

      3. Issue a new query before the network is reconnected.

      4. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.

      5. Reconnect the network.

      6. Connection attempt of SDK succeeds and the updates start coming for both the queries.










    share|improve this question


























      0












      0








      0








      I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.



      Please help me in debugging this issue.



      Edit:



      Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:




      • Scenario 1:



        1. Watch for query updates

        2. Client sets up a GRPC connection with the server.

        3. Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server

        4. Disconnect the network

        5. Reconnect within a minute, the query connection is still active and document updates arrive as expected

        6. Disconnect the network

        7. Reconnect after 5 minutes

        8. Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.

        9. No document updates are received at the client side (as expected)

        10. Issue a new query and watch for its updates.

        11. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.



      • Scenario 2:



        1. Watch for query updates

        2. Disconnect the network for about 5 minutes

        3. Issue a new query before the network is reconnected.

        4. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.

        5. Reconnect the network.

        6. Connection attempt of SDK succeeds and the updates start coming for both the queries.










      share|improve this question
















      I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.



      Please help me in debugging this issue.



      Edit:



      Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:




      • Scenario 1:



        1. Watch for query updates

        2. Client sets up a GRPC connection with the server.

        3. Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server

        4. Disconnect the network

        5. Reconnect within a minute, the query connection is still active and document updates arrive as expected

        6. Disconnect the network

        7. Reconnect after 5 minutes

        8. Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.

        9. No document updates are received at the client side (as expected)

        10. Issue a new query and watch for its updates.

        11. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.



      • Scenario 2:



        1. Watch for query updates

        2. Disconnect the network for about 5 minutes

        3. Issue a new query before the network is reconnected.

        4. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.

        5. Reconnect the network.

        6. Connection attempt of SDK succeeds and the updates start coming for both the queries.







      java firebase google-cloud-firestore






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 15 '18 at 14:09







      Sudarshan Murthy

















      asked Nov 15 '18 at 6:44









      Sudarshan MurthySudarshan Murthy

      7711




      7711






















          2 Answers
          2






          active

          oldest

          votes


















          0














          When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.



          The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.



          As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.






          share|improve this answer























          • Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.

            – Sudarshan Murthy
            Nov 15 '18 at 14:13











          • If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.

            – Alex Mamo
            Nov 15 '18 at 14:16












          • Thank you. I have raised an issue in the github tracker

            – Sudarshan Murthy
            Nov 15 '18 at 14:57











          • Ok, please keep me also posted. Thanks!

            – Alex Mamo
            Nov 15 '18 at 15:53











          • Looks like there is a setting to enable the GRPC keepalive in the FirestoreOptions while initializing the FirebaseApp. Please see my answer for details

            – Sudarshan Murthy
            Nov 19 '18 at 5:58


















          0














          The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
          This can be done by providing TransportChannelProvider in the FirestoreOptions while initializing the FirebaseApp.
          The code snippet is given below:



          InstantiatingGrpcChannelProvider channelProvider =
          InstantiatingGrpcChannelProvider.newBuilder()
          .setKeepAliveTime(Duration.ofSeconds(60L))
          .setKeepAliveTimeout(Duration.ofMinutes(5L))
          .build();

          FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
          .setChannelProvider(channelProvider).build();

          FirebaseOptions options = new FirebaseOptions.Builder()
          .setCredentials(credentials).setFirestoreOptions(firestoreOptions)
          .setConnectTimeout(5000).setReadTimeout(5000).build();
          FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);

          Firestore firestore = FirestoreClient.getFirestore(firebaseApp);





          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function ()
            StackExchange.using("externalEditor", function ()
            StackExchange.using("snippets", function ()
            StackExchange.snippets.init();
            );
            );
            , "code-snippets");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "1"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader:
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            ,
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );













            draft saved

            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53313824%2fnot-receiving-document-updates-on-network-disconnect-and-reconnect-in-firestore%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.



            The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.



            As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.






            share|improve this answer























            • Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.

              – Sudarshan Murthy
              Nov 15 '18 at 14:13











            • If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.

              – Alex Mamo
              Nov 15 '18 at 14:16












            • Thank you. I have raised an issue in the github tracker

              – Sudarshan Murthy
              Nov 15 '18 at 14:57











            • Ok, please keep me also posted. Thanks!

              – Alex Mamo
              Nov 15 '18 at 15:53











            • Looks like there is a setting to enable the GRPC keepalive in the FirestoreOptions while initializing the FirebaseApp. Please see my answer for details

              – Sudarshan Murthy
              Nov 19 '18 at 5:58















            0














            When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.



            The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.



            As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.






            share|improve this answer























            • Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.

              – Sudarshan Murthy
              Nov 15 '18 at 14:13











            • If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.

              – Alex Mamo
              Nov 15 '18 at 14:16












            • Thank you. I have raised an issue in the github tracker

              – Sudarshan Murthy
              Nov 15 '18 at 14:57











            • Ok, please keep me also posted. Thanks!

              – Alex Mamo
              Nov 15 '18 at 15:53











            • Looks like there is a setting to enable the GRPC keepalive in the FirestoreOptions while initializing the FirebaseApp. Please see my answer for details

              – Sudarshan Murthy
              Nov 19 '18 at 5:58













            0












            0








            0







            When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.



            The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.



            As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.






            share|improve this answer













            When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.



            The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.



            As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Nov 15 '18 at 9:33









            Alex MamoAlex Mamo

            47k82965




            47k82965












            • Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.

              – Sudarshan Murthy
              Nov 15 '18 at 14:13











            • If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.

              – Alex Mamo
              Nov 15 '18 at 14:16












            • Thank you. I have raised an issue in the github tracker

              – Sudarshan Murthy
              Nov 15 '18 at 14:57











            • Ok, please keep me also posted. Thanks!

              – Alex Mamo
              Nov 15 '18 at 15:53











            • Looks like there is a setting to enable the GRPC keepalive in the FirestoreOptions while initializing the FirebaseApp. Please see my answer for details

              – Sudarshan Murthy
              Nov 19 '18 at 5:58

















            • Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.

              – Sudarshan Murthy
              Nov 15 '18 at 14:13











            • If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.

              – Alex Mamo
              Nov 15 '18 at 14:16












            • Thank you. I have raised an issue in the github tracker

              – Sudarshan Murthy
              Nov 15 '18 at 14:57











            • Ok, please keep me also posted. Thanks!

              – Alex Mamo
              Nov 15 '18 at 15:53











            • Looks like there is a setting to enable the GRPC keepalive in the FirestoreOptions while initializing the FirebaseApp. Please see my answer for details

              – Sudarshan Murthy
              Nov 19 '18 at 5:58
















            Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.

            – Sudarshan Murthy
            Nov 15 '18 at 14:13





            Actually I have waited for more than an hour to see if the SDK retries and starts getting the updates but it didn't happen. This looks to be SDK issue. Please see my edit to the question. I have added some observations from the experiments that I tried with SDK logs enabled.

            – Sudarshan Murthy
            Nov 15 '18 at 14:13













            If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.

            – Alex Mamo
            Nov 15 '18 at 14:16






            If you have other issues regarding the way Firebase SDK work which cannot be explained using the docs or answers from SOF, I think you should consider contacting Firebase team.

            – Alex Mamo
            Nov 15 '18 at 14:16














            Thank you. I have raised an issue in the github tracker

            – Sudarshan Murthy
            Nov 15 '18 at 14:57





            Thank you. I have raised an issue in the github tracker

            – Sudarshan Murthy
            Nov 15 '18 at 14:57













            Ok, please keep me also posted. Thanks!

            – Alex Mamo
            Nov 15 '18 at 15:53





            Ok, please keep me also posted. Thanks!

            – Alex Mamo
            Nov 15 '18 at 15:53













            Looks like there is a setting to enable the GRPC keepalive in the FirestoreOptions while initializing the FirebaseApp. Please see my answer for details

            – Sudarshan Murthy
            Nov 19 '18 at 5:58





            Looks like there is a setting to enable the GRPC keepalive in the FirestoreOptions while initializing the FirebaseApp. Please see my answer for details

            – Sudarshan Murthy
            Nov 19 '18 at 5:58













            0














            The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
            This can be done by providing TransportChannelProvider in the FirestoreOptions while initializing the FirebaseApp.
            The code snippet is given below:



            InstantiatingGrpcChannelProvider channelProvider =
            InstantiatingGrpcChannelProvider.newBuilder()
            .setKeepAliveTime(Duration.ofSeconds(60L))
            .setKeepAliveTimeout(Duration.ofMinutes(5L))
            .build();

            FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
            .setChannelProvider(channelProvider).build();

            FirebaseOptions options = new FirebaseOptions.Builder()
            .setCredentials(credentials).setFirestoreOptions(firestoreOptions)
            .setConnectTimeout(5000).setReadTimeout(5000).build();
            FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);

            Firestore firestore = FirestoreClient.getFirestore(firebaseApp);





            share|improve this answer



























              0














              The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
              This can be done by providing TransportChannelProvider in the FirestoreOptions while initializing the FirebaseApp.
              The code snippet is given below:



              InstantiatingGrpcChannelProvider channelProvider =
              InstantiatingGrpcChannelProvider.newBuilder()
              .setKeepAliveTime(Duration.ofSeconds(60L))
              .setKeepAliveTimeout(Duration.ofMinutes(5L))
              .build();

              FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
              .setChannelProvider(channelProvider).build();

              FirebaseOptions options = new FirebaseOptions.Builder()
              .setCredentials(credentials).setFirestoreOptions(firestoreOptions)
              .setConnectTimeout(5000).setReadTimeout(5000).build();
              FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);

              Firestore firestore = FirestoreClient.getFirestore(firebaseApp);





              share|improve this answer

























                0












                0








                0







                The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
                This can be done by providing TransportChannelProvider in the FirestoreOptions while initializing the FirebaseApp.
                The code snippet is given below:



                InstantiatingGrpcChannelProvider channelProvider =
                InstantiatingGrpcChannelProvider.newBuilder()
                .setKeepAliveTime(Duration.ofSeconds(60L))
                .setKeepAliveTimeout(Duration.ofMinutes(5L))
                .build();

                FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
                .setChannelProvider(channelProvider).build();

                FirebaseOptions options = new FirebaseOptions.Builder()
                .setCredentials(credentials).setFirestoreOptions(firestoreOptions)
                .setConnectTimeout(5000).setReadTimeout(5000).build();
                FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);

                Firestore firestore = FirestoreClient.getFirestore(firebaseApp);





                share|improve this answer













                The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side.
                This can be done by providing TransportChannelProvider in the FirestoreOptions while initializing the FirebaseApp.
                The code snippet is given below:



                InstantiatingGrpcChannelProvider channelProvider =
                InstantiatingGrpcChannelProvider.newBuilder()
                .setKeepAliveTime(Duration.ofSeconds(60L))
                .setKeepAliveTimeout(Duration.ofMinutes(5L))
                .build();

                FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
                .setChannelProvider(channelProvider).build();

                FirebaseOptions options = new FirebaseOptions.Builder()
                .setCredentials(credentials).setFirestoreOptions(firestoreOptions)
                .setConnectTimeout(5000).setReadTimeout(5000).build();
                FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);

                Firestore firestore = FirestoreClient.getFirestore(firebaseApp);






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 19 '18 at 5:56









                Sudarshan MurthySudarshan Murthy

                7711




                7711



























                    draft saved

                    draft discarded
















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid


                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.

                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53313824%2fnot-receiving-document-updates-on-network-disconnect-and-reconnect-in-firestore%23new-answer', 'question_page');

                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Use pre created SQLite database for Android project in kotlin

                    Darth Vader #20

                    Ondo