User Access Control on HDFS through WebHDFS API











up vote
0
down vote

favorite












Is it possible to have user access control on file level of HDFS?
Currently, everyone can access the data when clicking on the link like http://<domain>/webhdfs/v1/xxx.txt?op=OPEN.



Can I set certain groups of people and they can only access certain files or directories? The purpose is, not letting others access files which don't belong to them.



If this cannot be set on HDFS, is there any suggestion?






hadoop hdfs







share|improve this question


























    up vote
    0
    down vote

    favorite












    Is it possible to have user access control on file level of HDFS?
    Currently, everyone can access the data when clicking on the link like http://<domain>/webhdfs/v1/xxx.txt?op=OPEN.



    Can I set certain groups of people and they can only access certain files or directories? The purpose is, not letting others access files which don't belong to them.



    If this cannot be set on HDFS, is there any suggestion?






    hadoop hdfs







    share|improve this question
























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      Is it possible to have user access control on file level of HDFS?
      Currently, everyone can access the data when clicking on the link like http://<domain>/webhdfs/v1/xxx.txt?op=OPEN.



      Can I set certain groups of people and they can only access certain files or directories? The purpose is, not letting others access files which don't belong to them.



      If this cannot be set on HDFS, is there any suggestion?






      hadoop hdfs







      share|improve this question













      Is it possible to have user access control on file level of HDFS?
      Currently, everyone can access the data when clicking on the link like http://<domain>/webhdfs/v1/xxx.txt?op=OPEN.



      Can I set certain groups of people and they can only access certain files or directories? The purpose is, not letting others access files which don't belong to them.



      If this cannot be set on HDFS, is there any suggestion?






      hadoop hdfs




      hadoop hdfs






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 7 hours ago









      Ryan Yang

      265




      265
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          1
          down vote













          When security is off, the authenticated user is the username specified in the user.name query parameter. If the user.name parameter is not set, the server may either set the authenticated user to a default web user, if there is any, or return an error response.



          When security is on, authentication is performed by either Hadoop delegation token or Kerberos SPNEGO. If a token is set in the delegation query parameter, the authenticated user is the user encoded in the token. If the delegation parameter is not set, the user is authenticated by Kerberos SPNEGO.



          Below are examples using the curl command tool.



          Authentication when security is off:



          curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?[user.name=<USER>&]op=..."


          Authentication using Kerberos SPNEGO when security is on:



          curl -i --negotiate -u : "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=..."


          Authentication using Hadoop delegation token when security is on:



          curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?delegation=<TOKEN>&op=..."


          Ref: https://hadoop.apache.org/docs/r2.9.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Authentication






          share|improve this answer





















            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53370469%2fuser-access-control-on-hdfs-through-webhdfs-api%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            1
            down vote













            When security is off, the authenticated user is the username specified in the user.name query parameter. If the user.name parameter is not set, the server may either set the authenticated user to a default web user, if there is any, or return an error response.



            When security is on, authentication is performed by either Hadoop delegation token or Kerberos SPNEGO. If a token is set in the delegation query parameter, the authenticated user is the user encoded in the token. If the delegation parameter is not set, the user is authenticated by Kerberos SPNEGO.



            Below are examples using the curl command tool.



            Authentication when security is off:



            curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?[user.name=<USER>&]op=..."


            Authentication using Kerberos SPNEGO when security is on:



            curl -i --negotiate -u : "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=..."


            Authentication using Hadoop delegation token when security is on:



            curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?delegation=<TOKEN>&op=..."


            Ref: https://hadoop.apache.org/docs/r2.9.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Authentication






            share|improve this answer

























              up vote
              1
              down vote













              When security is off, the authenticated user is the username specified in the user.name query parameter. If the user.name parameter is not set, the server may either set the authenticated user to a default web user, if there is any, or return an error response.



              When security is on, authentication is performed by either Hadoop delegation token or Kerberos SPNEGO. If a token is set in the delegation query parameter, the authenticated user is the user encoded in the token. If the delegation parameter is not set, the user is authenticated by Kerberos SPNEGO.



              Below are examples using the curl command tool.



              Authentication when security is off:



              curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?[user.name=<USER>&]op=..."


              Authentication using Kerberos SPNEGO when security is on:



              curl -i --negotiate -u : "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=..."


              Authentication using Hadoop delegation token when security is on:



              curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?delegation=<TOKEN>&op=..."


              Ref: https://hadoop.apache.org/docs/r2.9.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Authentication






              share|improve this answer























                up vote
                1
                down vote










                up vote
                1
                down vote









                When security is off, the authenticated user is the username specified in the user.name query parameter. If the user.name parameter is not set, the server may either set the authenticated user to a default web user, if there is any, or return an error response.



                When security is on, authentication is performed by either Hadoop delegation token or Kerberos SPNEGO. If a token is set in the delegation query parameter, the authenticated user is the user encoded in the token. If the delegation parameter is not set, the user is authenticated by Kerberos SPNEGO.



                Below are examples using the curl command tool.



                Authentication when security is off:



                curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?[user.name=<USER>&]op=..."


                Authentication using Kerberos SPNEGO when security is on:



                curl -i --negotiate -u : "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=..."


                Authentication using Hadoop delegation token when security is on:



                curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?delegation=<TOKEN>&op=..."


                Ref: https://hadoop.apache.org/docs/r2.9.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Authentication






                share|improve this answer












                When security is off, the authenticated user is the username specified in the user.name query parameter. If the user.name parameter is not set, the server may either set the authenticated user to a default web user, if there is any, or return an error response.



                When security is on, authentication is performed by either Hadoop delegation token or Kerberos SPNEGO. If a token is set in the delegation query parameter, the authenticated user is the user encoded in the token. If the delegation parameter is not set, the user is authenticated by Kerberos SPNEGO.



                Below are examples using the curl command tool.



                Authentication when security is off:



                curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?[user.name=<USER>&]op=..."


                Authentication using Kerberos SPNEGO when security is on:



                curl -i --negotiate -u : "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=..."


                Authentication using Hadoop delegation token when security is on:



                curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?delegation=<TOKEN>&op=..."


                Ref: https://hadoop.apache.org/docs/r2.9.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Authentication







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered 5 hours ago









                Jeba

                714




                714






























                     

                    draft saved


                    draft discarded



















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53370469%2fuser-access-control-on-hdfs-through-webhdfs-api%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    Volksrepublik China

                    Image
                    Dieser Artikel befasst sich mit der Volksrepublik China und ist nicht zu verwechseln mit der Republik China (Taiwan). Siehe auch China (Begriffsklärung). Dieser Artikel wurde aufgrund von inhaltlichen Mängeln auf der Qualitätssicherungsseite der Redaktion Ostasien eingetragen. Dies geschieht, um die Qualität der Artikel aus dem Themengebiet Ostasien auf ein akzeptables Niveau zu bringen. Dabei werden Artikel gelöscht, die nicht signifikant verbessert werden können. Bitte hilf mit, die inhaltlichen Mängel dieses Artikels zu beseitigen, und beteilige dich bitte an der Diskussion ! 中华人民共和国 Zhōnghuá Rénmín Gònghéguó Volksrepublik China Flagge Emblem Amtssprache Chinesisch (Hochchinesisch) und anerkannte Regionalsprachen 1 Hauptstadt Peking Staatsform Republik Regierungssystem Sozialistisches Einparteiensystem Staatsoberhaupt Staatspräsident Xi Jinping Regierungschef Ministerpräsident Li Keqiang Fläc...
                    Read more

                    How to test boost logger output in unit testing?

                    Image
                    up vote 1 down vote favorite My application uses BOOST_LOG_TRIVIAL to log messages. I thought it's good to check in test (gtest) that correct messages are written there in given scenarios. Is there a way to somehow access what was written there after code was called? Or does it first have to be mocked in some way? I have googled a lot for this and haven't found any instructions, so either I am asking wrong questions or no one thinks it should be done. c++ logging boost googletest boost-log share | improve this question edited yesterday ...
                    Read more

                    Write to the output between two pipeline

                    Image
                    up vote 1 down vote favorite I'm trying to write to the output (echo) within two pipe to have a trace of what I'm doing but I have an error. Get-ChildItem $path -Recurse | Where-Object { $_.Name -match '.+?.log.(d{4})-(d{2})-(d{2})$' -and $_.LastWriteTime -lt $deleteDate } | Write-Output "deleting file" -PassThru | Remove-Item but I have this error : Write-Output : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any of the parameters that take pipeline input. I also tested in the Foreach-Object function Get-ChildItem $path -Recurse | Where-Object { $_.Name -match '.+?.log.(d{4})-(d{2})-(d{2})$' -and $_.LastWriteTime -lt $deleteDate } | Foreach-Object...
                    Read more