add authentication to fluentd instance getting syslogs from cloud foundry application









up vote
0
down vote

favorite












I have a cloud foundry application and I had a requirement to get the logs into an S3 bucket.



I followed the advice here and have used a docker image to push onto a cloud foundry application. Logs now go to the fluentd instance and onto S3.



The problem I have is that the fluentd instance is publicly visible and the cloud foundry provider has not implemented this so we cannot enable container communication.



I can do this



curl -X POST -d 'json="foo":"bar"' https://fluentd.my.domain



And it will record a log.



Is there anyway I can somehow limit traffic to the fluentd instance.



I don't see how I can add a custom header or anything when cloud foundry is sending the syslog.






cloudfoundry fluentd







share|improve this question





















  • Your link shows it is accepting syslog input over TCP/UDP but your curl command is trying to push data over HTTP protocol. Question is, are you using syslog input or http input at fluentd source?. Can you paste your fluentd full config as well?>
    – Imran
    Nov 9 at 21:04










  • I am outside of work unfortunately and I do not have the config available but it is not TCP/UDP it is http. If it is cloud foundry app to cloud foundry app then it is http. The curl works
    – dagda1
    Nov 9 at 21:29










  • It appears, HTTP Input does not support any auth. stackoverflow.com/questions/38146565/…. if you are using syslog input then we could enable mutual auth between your application and fluentd. Let me know if this helps then I will put it as answer with more details on secure syslog.
    – Imran
    Nov 10 at 2:23










  • @Imran syslog won't work, I've been down this path before. I had a syslog input but it would not connect. The reason is The documentation from Cloud Foundry on sending things to Fluentd doesn't appear to be running Fluentd on Cloud Foundry which is why it's talking regular syslog. Currently, anything expecting requests on our cloud foundry provider must be talking HTTP. I've clarified this with the support
    – dagda1
    Nov 10 at 12:26











  • When I changed from syslog input to the http input then it worked for the reason I mentioned above.
    – dagda1
    Nov 10 at 12:26















up vote
0
down vote

favorite












I have a cloud foundry application and I had a requirement to get the logs into an S3 bucket.



I followed the advice here and have used a docker image to push onto a cloud foundry application. Logs now go to the fluentd instance and onto S3.



The problem I have is that the fluentd instance is publicly visible and the cloud foundry provider has not implemented this so we cannot enable container communication.



I can do this



curl -X POST -d 'json="foo":"bar"' https://fluentd.my.domain



And it will record a log.



Is there anyway I can somehow limit traffic to the fluentd instance.



I don't see how I can add a custom header or anything when cloud foundry is sending the syslog.






cloudfoundry fluentd







share|improve this question





















  • Your link shows it is accepting syslog input over TCP/UDP but your curl command is trying to push data over HTTP protocol. Question is, are you using syslog input or http input at fluentd source?. Can you paste your fluentd full config as well?>
    – Imran
    Nov 9 at 21:04










  • I am outside of work unfortunately and I do not have the config available but it is not TCP/UDP it is http. If it is cloud foundry app to cloud foundry app then it is http. The curl works
    – dagda1
    Nov 9 at 21:29










  • It appears, HTTP Input does not support any auth. stackoverflow.com/questions/38146565/…. if you are using syslog input then we could enable mutual auth between your application and fluentd. Let me know if this helps then I will put it as answer with more details on secure syslog.
    – Imran
    Nov 10 at 2:23










  • @Imran syslog won't work, I've been down this path before. I had a syslog input but it would not connect. The reason is The documentation from Cloud Foundry on sending things to Fluentd doesn't appear to be running Fluentd on Cloud Foundry which is why it's talking regular syslog. Currently, anything expecting requests on our cloud foundry provider must be talking HTTP. I've clarified this with the support
    – dagda1
    Nov 10 at 12:26











  • When I changed from syslog input to the http input then it worked for the reason I mentioned above.
    – dagda1
    Nov 10 at 12:26













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I have a cloud foundry application and I had a requirement to get the logs into an S3 bucket.



I followed the advice here and have used a docker image to push onto a cloud foundry application. Logs now go to the fluentd instance and onto S3.



The problem I have is that the fluentd instance is publicly visible and the cloud foundry provider has not implemented this so we cannot enable container communication.



I can do this



curl -X POST -d 'json="foo":"bar"' https://fluentd.my.domain



And it will record a log.



Is there anyway I can somehow limit traffic to the fluentd instance.



I don't see how I can add a custom header or anything when cloud foundry is sending the syslog.






cloudfoundry fluentd







share|improve this question













I have a cloud foundry application and I had a requirement to get the logs into an S3 bucket.



I followed the advice here and have used a docker image to push onto a cloud foundry application. Logs now go to the fluentd instance and onto S3.



The problem I have is that the fluentd instance is publicly visible and the cloud foundry provider has not implemented this so we cannot enable container communication.



I can do this



curl -X POST -d 'json="foo":"bar"' https://fluentd.my.domain



And it will record a log.



Is there anyway I can somehow limit traffic to the fluentd instance.



I don't see how I can add a custom header or anything when cloud foundry is sending the syslog.






cloudfoundry fluentd




cloudfoundry fluentd






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 9 at 20:25









dagda1

9,93230129257




9,93230129257











  • Your link shows it is accepting syslog input over TCP/UDP but your curl command is trying to push data over HTTP protocol. Question is, are you using syslog input or http input at fluentd source?. Can you paste your fluentd full config as well?>
    – Imran
    Nov 9 at 21:04










  • I am outside of work unfortunately and I do not have the config available but it is not TCP/UDP it is http. If it is cloud foundry app to cloud foundry app then it is http. The curl works
    – dagda1
    Nov 9 at 21:29










  • It appears, HTTP Input does not support any auth. stackoverflow.com/questions/38146565/…. if you are using syslog input then we could enable mutual auth between your application and fluentd. Let me know if this helps then I will put it as answer with more details on secure syslog.
    – Imran
    Nov 10 at 2:23










  • @Imran syslog won't work, I've been down this path before. I had a syslog input but it would not connect. The reason is The documentation from Cloud Foundry on sending things to Fluentd doesn't appear to be running Fluentd on Cloud Foundry which is why it's talking regular syslog. Currently, anything expecting requests on our cloud foundry provider must be talking HTTP. I've clarified this with the support
    – dagda1
    Nov 10 at 12:26











  • When I changed from syslog input to the http input then it worked for the reason I mentioned above.
    – dagda1
    Nov 10 at 12:26

















  • Your link shows it is accepting syslog input over TCP/UDP but your curl command is trying to push data over HTTP protocol. Question is, are you using syslog input or http input at fluentd source?. Can you paste your fluentd full config as well?>
    – Imran
    Nov 9 at 21:04










  • I am outside of work unfortunately and I do not have the config available but it is not TCP/UDP it is http. If it is cloud foundry app to cloud foundry app then it is http. The curl works
    – dagda1
    Nov 9 at 21:29










  • It appears, HTTP Input does not support any auth. stackoverflow.com/questions/38146565/…. if you are using syslog input then we could enable mutual auth between your application and fluentd. Let me know if this helps then I will put it as answer with more details on secure syslog.
    – Imran
    Nov 10 at 2:23










  • @Imran syslog won't work, I've been down this path before. I had a syslog input but it would not connect. The reason is The documentation from Cloud Foundry on sending things to Fluentd doesn't appear to be running Fluentd on Cloud Foundry which is why it's talking regular syslog. Currently, anything expecting requests on our cloud foundry provider must be talking HTTP. I've clarified this with the support
    – dagda1
    Nov 10 at 12:26











  • When I changed from syslog input to the http input then it worked for the reason I mentioned above.
    – dagda1
    Nov 10 at 12:26
















Your link shows it is accepting syslog input over TCP/UDP but your curl command is trying to push data over HTTP protocol. Question is, are you using syslog input or http input at fluentd source?. Can you paste your fluentd full config as well?>
– Imran
Nov 9 at 21:04




Your link shows it is accepting syslog input over TCP/UDP but your curl command is trying to push data over HTTP protocol. Question is, are you using syslog input or http input at fluentd source?. Can you paste your fluentd full config as well?>
– Imran
Nov 9 at 21:04












I am outside of work unfortunately and I do not have the config available but it is not TCP/UDP it is http. If it is cloud foundry app to cloud foundry app then it is http. The curl works
– dagda1
Nov 9 at 21:29




I am outside of work unfortunately and I do not have the config available but it is not TCP/UDP it is http. If it is cloud foundry app to cloud foundry app then it is http. The curl works
– dagda1
Nov 9 at 21:29












It appears, HTTP Input does not support any auth. stackoverflow.com/questions/38146565/…. if you are using syslog input then we could enable mutual auth between your application and fluentd. Let me know if this helps then I will put it as answer with more details on secure syslog.
– Imran
Nov 10 at 2:23




It appears, HTTP Input does not support any auth. stackoverflow.com/questions/38146565/…. if you are using syslog input then we could enable mutual auth between your application and fluentd. Let me know if this helps then I will put it as answer with more details on secure syslog.
– Imran
Nov 10 at 2:23












@Imran syslog won't work, I've been down this path before. I had a syslog input but it would not connect. The reason is The documentation from Cloud Foundry on sending things to Fluentd doesn't appear to be running Fluentd on Cloud Foundry which is why it's talking regular syslog. Currently, anything expecting requests on our cloud foundry provider must be talking HTTP. I've clarified this with the support
– dagda1
Nov 10 at 12:26





@Imran syslog won't work, I've been down this path before. I had a syslog input but it would not connect. The reason is The documentation from Cloud Foundry on sending things to Fluentd doesn't appear to be running Fluentd on Cloud Foundry which is why it's talking regular syslog. Currently, anything expecting requests on our cloud foundry provider must be talking HTTP. I've clarified this with the support
– dagda1
Nov 10 at 12:26













When I changed from syslog input to the http input then it worked for the reason I mentioned above.
– dagda1
Nov 10 at 12:26





When I changed from syslog input to the http input then it worked for the reason I mentioned above.
– dagda1
Nov 10 at 12:26


















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53232826%2fadd-authentication-to-fluentd-instance-getting-syslogs-from-cloud-foundry-applic%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53232826%2fadd-authentication-to-fluentd-instance-getting-syslogs-from-cloud-foundry-applic%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

Kleinkühnau

Image
Kleinkühnau Stadt Dessau-Roßlau 51.839444444444 12.184444444444 57 Koordinaten: 51° 50′ 22″  N , 12° 11′ 4″  O Höhe: 57 m Fläche: 9,93 km² Einwohner: 1649  (31. Dez. 2011) Bevölkerungsdichte: 166 Einwohner/km² Eingemeindung: 1. Oktober 1923 Eingemeindet nach: Dessau Postleitzahl: 06846 Vorwahl: 0340 Rathaus (Amtshaus Kühnau e. V.) Ev. Kirche Kleinkühnau Kleinkühnau ist ein Stadtteil von Dessau-Roßlau, einer kreisfreien Stadt im Bundesland Sachsen-Anhalt. Er liegt ungefähr vier Kilometer westlich des Stadtzentrums. Südlich der Kühnauer Straße befinden sich der Flugplatz Dessau, das Technikmuseum Hugo Junkers sowie der Standort der Polizeidirektion Sachsen-Anhalt Ost. Kleinkühnau hat 1649 Einwohner (Stand 31. Dezember 2011). Inhaltsverzeichnis 1 Geschichte 1.1 Bevölkerungsentwicklung 2 Siehe auch 3 Weblinks 4 E...
Read more

Makov (Slowakei)

Image
Makov Wappen Karte Makov Basisdaten Staat: Slowakei Kraj: Žilinský kraj Okres: Čadca Region: Kysuce Fläche: 46,052 km² Einwohner: 1.720 (31. Dez. 2017) Bevölkerungsdichte: 37 Einwohner je km² Höhe: 583  m n.m. Postleitzahl: 023 56 Telefonvorwahl: 0 41 Geographische Lage: 49° 22′  N , 18° 29′  O 49.366666666667 18.483333333333 583 Koordinaten: 49° 22′ 0″  N , 18° 29′ 0″  O Kfz-Kennzeichen: CA Kód obce: 509299 Struktur Gemeindeart: Gemeinde Verwaltung (Stand: November 2014) Bürgermeister: Martin Pavlík Adresse: Obecný úrad Makov 60 02356 Makov Webpräsenz: www.makov.sk Statistikinformation auf statistics.sk Makov (ungarisch Trencsénmakó – älter auch Makó , polnisch Maków ) ist eine Gemeinde in der Nordwestslowakei. Inhaltsverzeichnis 1 Geographie 1.1 Bevölkerung 2 Ge...
Read more

Deutsches Schauspielhaus

Image
Deutsches Schauspielhaus Hamburg Aktie über 1000 Mark der Deutsche Schauspielhaus AG vom 20. Juni 1899 Das Deutsche Schauspielhaus im Hamburger Stadtteil St. Georg ist mit 1200 Plätzen das größte Sprechtheater Deutschlands. Entstanden ist es durch eine private Initiative von Bürgern und der 1899 gegründeten „Aktiengesellschaft Deutsches Schauspielhaus“. Die Pläne stammen von dem Wiener Architekturbüro Fellner und Helmer, die das neobarocke Gebäude nach dem Vorbild des Wiener Volkstheaters gestalteten. Am 15. September 1900 wurde das Theater mit einer Aufführung der Iphigenie auf Tauris feierlich eröffnet. Der Name Deutsches Schauspielhaus knüpft an die Tradition des Hamburger Nationaltheaters an, das Mitte des 18. Jahrhunderts durch Hamburger Bürger betrieben wurde und an dem Gotthold Ephraim Lessing als Dramaturg tätig war. Das Schauspielhaus hat die Theatergeschichte der Stadt im 20. Jahrhundert entsc...
Read more