内容简介：greenplum 集群 迷雾重重
报错 1 error whileloading shared libraries: libyaml-0.so.1
在单独的 linux 服务器安装gpfdist之后，运行报错：
[gpadmin@g01~]$ gpfdist
gpfdist: error while loading shared libraries: libyaml-0.so.1: cannot open shared object file: No such file or directory
[gpadmin@g01~]$
/data/greenplum/bin/gpfdist: error while loading shared libraries: libyaml-0.so.1: cannot open shared object file: No such file or directory
[gpadmin@g01~]$
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
查看，已经存在了，只是不识别而已：
[gpadmin@g01~]$ find /data/greenplum/ -name libyaml-0.so.1
/data/greenplum/lib/libyaml-0.so.1
[gpadmin@g01~]$
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Google到一篇文章 https://discuss.pivotal.io/hc/en-us/articles/218036377-gpfdist-shows-error-message-while-loading-shared-libraries-libyaml-0-so-1-cannot-open-shared-object-file-No-such-file-or-directory- ，说可能是版本比较低引起的
Cause
As explained in the 4.3.7.x release note shownbelow, the library file libyaml-0.so.1 was missed in the releases prior to4.3.7.0.
|
26075 |
Loaders: gpload |
4.3.7.0 |
The Greenplum Database Loaders for Red Hat Enterprise 5 and 6 did not contain the library libyaml-0.so.1 . This issue has been resolved. |
查看安装的loaders版本信息：
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[gpadmin@g01lib]$ ll /home/gpadmin/
total 29200
-rw-rw-r-- 1 gpadmin gpadmin 163 May 11 17:43 1
-rw-rw-r-- 1 gpadmin gpadmin 141 May 12 09:50 gpfdist.log
-rwxr-xr-x 1 gpadmin gpadmin 15027400 May 10 2016 greenplum-loaders-4.3.8.2-build-1-RHEL5-x86_64.bin
-rw-r--r-- 1 gpadmin gpadmin 14863550 May 11 16:43 greenplum-loaders-4.3.8.2-build-1-RHEL5-x86_64.zip
[gpadmin@g01lib]$
[gpadmin@g01lib]$ ll /data/greenplum/lib/libyaml*
lrwxrwxrwx 1 gpadmin gpadmin 18 May 11 17:17 /data/greenplum/lib/libyaml-0.so.1 -> libyaml-0.so.1.0.0
-r-xr-xr-x 1 gpadmin gpadmin 319651 Feb 1 2014 /data/greenplum/lib/libyaml-0.so.1.0.0
lrwxrwxrwx 1 gpadmin gpadmin 18 May 11 17:17 /data/greenplum/lib/libyaml.so -> libyaml-0.so.1.0.0
[gpadmin@g01lib]$
看版本是已经是4.3.8.x了，而且lib目录下也已经有了libyaml的版本了，只是不识别，看来不是版本过低的问题导致的。
继续google了下，查看到文章 https://discuss.pivotal.io/hc/en-us/community/posts/206003588-Load-Tools-error-while-loading-shared-libraries-libyaml-0-so-1 上面dillon网友有说添加一个lib的path路径 LD_LIBRARY_PATH ，尝试，问题解决，不会报错了。
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[gpadmin@g01~]$ vim .bash_profile
……
export GPHOME=/data/greenplum
export LD_LIBRARY_PATH=/data/greenplum/lib
PATH=$PATH:$HOME/bin:$GPHOME/bin
#export GPHOME=/data/greenplum
export PATH
[gpadmin@g01~]$ gpfdist
2017-05-12 13:48:53 13870 INFO Before opening listening sockets - following listening sockets are available:
2017-05-12 13:48:53 13870 INFO IPV6 socket: [::]:8080
2017-05-12 13:48:53 13870 INFO IPV4 socket: 0.0.0.0:8080
2017-05-12 13:48:53 13870 INFO Trying to open listening socket:
2017-05-12 13:48:53 13870 INFO IPV6 socket: [::]:8080
2017-05-12 13:48:53 13870 INFO Opening listening socket succeeded
2017-05-12 13:48:53 13870 INFO Trying to open listening socket:
2017-05-12 13:48:53 13870 INFO IPV4 socket: 0.0.0.0:8080
Serving HTTP on port 8080, directory /home/gpadmin
报错 2 不识别 greenplum_path.sh
[gpadmin@g01~]$ gpload -f gpload.yml
/data/greenplum/bin/gpload: line 3:/data/greenplum/greenplum_path.sh: No such file or directory
[gpadmin@g01~]$
解决：自己重建一个
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@g01~]# more /data/greenplum/greenplum_path.sh
GPHOME=/data/greenplum
# Replace with symlink path if it is present and correct
if [ -h ${GPHOME}/../greenplum-db ]; then
GPHOME_BY_SYMLINK=`(cd ${GPHOME}/../greenplum-db/ && pwd -P)`
if [ x"${GPHOME_BY_SYMLINK}" = x"${GPHOME}" ]; then
GPHOME=`(cd ${GPHOME}/../greenplum-db/ && pwd -L)`/.
fi
unset GPHOME_BY_SYMLINK
fi
#setup PYTHONHOME
if [ -x $GPHOME/ext/python/bin/python ]; then
PYTHONHOME="$GPHOME/ext/python"
fi
PYTHONPATH=$GPHOME/lib/python
PATH=$GPHOME/bin:$PYTHONHOME/bin:$PATH
LD_LIBRARY_PATH=$GPHOME/lib:$PYTHONHOME/lib:$LD_LIBRARY_PATH
OPENSSL_CONF=$GPHOME/etc/openssl.cnf
export GPHOME
export PATH
export LD_LIBRARY_PATH
export PYTHONPATH
export PYTHONHOME
export OPENSSL_CONF
[root@g01~]#
blog源地址： http://blog.csdn.net/mchdba/article/details/72848708 ，作者mchdba 黄杉，谢绝转载。
报错 3 gpload 加载失败
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[gpadmin@g01~]$ gpload -f gpload.yml
2017-05-12 16:15:01|INFO|gpload session started 2017-05-12 16:15:01
2017-05-12 16:15:01|ERROR|could not connect to database: FATAL: no pg_hba.conf entry for host "192.168.2.72", user "gpadmin", database "yueworld_db", SSL off
. Is the Greenplum Database running on port 5432?
2017-05-12 16:15:01|INFO|rows Inserted = 0
2017-05-12 16:15:01|INFO|rows Updated = 0
2017-05-12 16:15:01|INFO|data formatting errors = 0
2017-05-12 16:15:01|INFO|gpload failed
[gpadmin@g01~]$
解决办法：gpload是在greenplum的数据库服务器上面执行的，不是在单独的gpfdist的服务器上执行。
报错 4 cannot changea readable external table
[gpadmin@dwhm01_2_111 ~]$ gpload -fgpload.yml
2017-05-12 16:57:41|INFO|gpload sessionstarted 2017-05-12 16:57:41
2017-05-12 16:57:41|INFO|setting schema'public' for table 't01'
2017-05-12 16:57:41|INFO|started gpfdist -p8090 -P 8091 -f "/home/gpadmin/gpdextdata/t01.txt/home/gpadmin/gpdextdata/t02.txt" -t 30
2017-05-12 16:57:42|ERROR|ERROR: cannot change a readable external table"t01"
encountered while running INSERT INTOpublic."t01" ("id","name") SELECT"id","name" FROMext_gpload_0e8b9f06_36f1_11e7_907a_0017fa009565
2017-05-12 16:57:42|INFO|rows Inserted = 0
2017-05-12 16:57:42|INFO|rows Updated = 0
2017-05-12 16:57:42|INFO|data formattingerrors = 0
2017-05-12 16:57:42|INFO|gpload failed
[gpadmin@dwhm01_2_111 ~]$
【问题解决】：需要提前建立的可读写的外部表，sql语句如：create writable external table public.t01……
报错 5 permissiondenied: no privilege
[gpadmin@g01~]$ gpload -f gpload.yml
2017-05-12 17:35:42|INFO|gpload session started 2017-05-12 17:35:42
2017-05-12 17:35:42|INFO|setting schema 'public' for table 't01'
2017-05-12 17:35:42|INFO|started gpfdist -p 8090 -P 8091 -f "/home/gpadmin/gpdextdata/t01.txt /home/gpadmin/gpdextdata/t02.txt" -t 30
2017-05-12 17:35:42|ERROR|could not run SQL "create external table ext_gpload_5e20945e_36f6_11e7_a5f0_0017fa004ce3(id int,name text)location('gpfdist://192.168.2.72:8091//home/gpadmin/gpdextdata/t01.txt%20/home/gpadmin/gpdextdata/t02.txt') format'text' (delimiter ',' null '\\N' escape '\\' ) segment reject limit 25 ": ERROR: permission denied: no privilege to create a readable gpfdist(s) external table
2017-05-12 17:35:42|INFO|rows Inserted = 0
2017-05-12 17:35:42|INFO|rows Updated = 0
2017-05-12 17:35:42|INFO|data formatting errors = 0
2017-05-12 17:35:42|INFO|gpload failed
[gpadmin@g01~]$
【解决办法：】
赋予ssb用户创建外部表权限：
要允许用户创建外部表，否则建外部表时会得到错误
ERROR: permission denied: no privilege to create a readable gpfdistexternal table
修改配置文件/data/greenplum-db-4.3.12.0/gpconfig/gpinitsystem_config，添加参数
gp_external_enable_exec = on # enable external tables withEXECUTE.
gp_external_grant_privileges = on #enable create http/gpfdist for non su's
允许非超级管理员创建外部表，必须重启数据库服务（使用gpstop –u命令该参数文件修改不生效），才能生效。
重启greenplum后，查看是否有权限：
yueworld_db=# show gp_external_enable_exec ;
gp_external_enable_exec
-------------------------
on
(1 row)
yueworld_db=#
yueworld_db=# show gp_external_grant_privileges;
gp_external_grant_privileges
------------------------------
off
(1 row)
yueworld_db=#
看到gp_external_grant_privileges还是off的，证明在gpinitsystem_config里面配置gp_external_grant_privileges无效的，所以直接使用命令行设置：
[gpadmin@dwhm01_2_111 gpconfig]$ gpconfig -c gp_external_grant_privileges -v on
20170515:10:50:05:017519 gpconfig:dwhm01_2_111:gpadmin-[INFO]:-completed successfully
[gpadmin@dwhm01_2_111 gpconfig]$ gpconfig -c gp_external_enable_exec -v on
20170515:10:50:12:017592 gpconfig:dwhm01_2_111:gpadmin-[INFO]:-completed successfully
[gpadmin@dwhm01_2_111 gpconfig]$
添加配置完后，再重启greenplum集群，生效：
[gpadmin@dwhm01_2_111 gpconfig]$ gpstop -r
20170515:10:51:08:017869 gpstop:dwhm01_2_111:gpadmin-[INFO]:-Starting gpstop with args: -r
20170515:10:51:08:017869 gpstop:dwhm01_2_111:gpadmin-[INFO]:-Gathering information and validating the environment...
20170515:10:51:08:017869 gpstop:dwhm01_2_111:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20170515:10:51:08:017869 gpstop:dwhm01_2_111:gpadmin-[INFO]:-Obtaining Segment details from master...
20170515:10:51:27:017869 gpstop:dwhm01_2_111:gpadmin-[INFO]:-Cleaning up leftover shared memory
20170515:10:51:33:017869 gpstop:dwhm01_2_111:gpadmin-[INFO]:-Restarting System...
[gpadmin@dwhm01_2_111 gpconfig]$
[gpadmin@dwhm01_2_111 gpconfig]$
[gpadmin@dwhm01_2_111 gpconfig]$ psql -d yueworld_db
psql (8.2.15)
Type "help" for help.
yueworld_db=# show gp_external_grant_privileges;
gp_external_grant_privileges
------------------------------
on
(1 row)
yueworld_db=# show gp_external_enable_exec ;
gp_external_enable_exec
-------------------------
on
(1 row)
yueworld_db=#
然后再执行数据导入操作，OK，不会报权限错误了：
[gpadmin@g01~]$ gpload -f gpload.yml
2017-05-15 10:53:47|INFO|gpload session started 2017-05-15 10:53:47
2017-05-15 10:53:47|INFO|setting schema 'public' for table 't01'
2017-05-15 10:53:47|INFO|started gpfdist -p 8090 -P 8091 -f "/home/gpadmin/gpdextdata/t01.txt /home/gpadmin/gpdextdata/t02.txt" -t 30
2017-05-15 10:53:48|INFO|running time: 0.37 seconds
2017-05-15 10:53:48|INFO|rows Inserted = 0
2017-05-15 10:53:48|INFO|rows Updated = 0
2017-05-15 10:53:48|INFO|data formatting errors = 6
2017-05-15 10:53:48|INFO|gpload succeeded
[gpadmin@g01~]$
报错 6 permissiondenied for schema dw
[gpadmin@g01~]$ gpload -f g2.yml
2017-05-19 17:02:38|INFO|gpload session started 2017-05-19 17:02:38
2017-05-19 17:02:38|INFO|started gpfdist -p 8090 -P 8091 -f "/data/greenplum/gpextdata/t21.txt /data/greenplum/gpextdata/t22.txt" -t 30
2017-05-19 17:02:38|ERROR|ERROR: permission denied for schema dw
encountered while running INSERT INTO "dw"."t02" ("id","name") SELECT "id","name" FROM ext_gpload_e8104a8a_3c71_11e7_bb89_0017fa004ce3
2017-05-19 17:02:38|INFO|rows Inserted = 0
2017-05-19 17:02:38|INFO|rows Updated = 0
2017-05-19 17:02:38|INFO|data formatting errors = 0
2017-05-19 17:02:38|INFO|gpload failed
[gpadmin@g01~]$
【解决方案】，在greenplum集群中，赋予mch账号权限。
先在greenplum集群的master上执行权限赋予操作
yueworld_db=# grant all on schema dw to mch;
GRANT
yueworld_db=#
然后去gpload服务器上执行
[gpadmin@g01~]$ gpload -f g2.yml
2017-05-19 17:09:59|INFO|gpload session started 2017-05-19 17:09:59
2017-05-19 17:09:59|INFO|started gpfdist -p 8090 -P 8091 -f "/data/greenplum/gpextdata/t21.txt /data/greenplum/gpextdata/t22.txt" -t 30
2017-05-19 17:09:59|INFO|running time: 0.32 seconds
2017-05-19 17:09:59|INFO|rows Inserted = 8
2017-05-19 17:09:59|INFO|rows Updated = 0
2017-05-19 17:09:59|INFO|data formatting errors = 0
2017-05-19 17:09:59|INFO|gpload succeeded
[gpadmin@g01~]$
以上就是本文的全部内容，希望本文的内容对大家的学习或者工作能带来一定的帮助，也希望大家多多支持 码农网
猜你喜欢:
- 战争迷雾开源库测评
- Python排序算法[二]：测试数据的迷雾散去
- 一个隐藏在黑客迷雾下的bug——记一次黑客攻防
- 识别迷雾中的物体，谷歌提出最新目标检测算法Context R-CNN
- Elasticsearch 集群搭建和集群原理
- Zookeeper集群 + Kafka集群 + KafkaOffsetMonitor 监控
本站部分资源来源于网络，本站转载出于传递更多信息之目的，版权归原作者或者来源机构所有，如转载稿涉及版权问题，请联系我们。
Beginning XML with DOM and Ajax
Sas Jacobs / Apress / 2006-06-05 / USD 39.99
Don't waste time on 1,000-page tomes full of syntax; this book is all you need to get ahead in XML development. Renowned web developer Sas Jacobs presents an essential guide to XML. Beginning XML with......一起来看看 《Beginning XML with DOM and Ajax》 这本书的介绍吧!
CSS 压缩/解压工具
在线压缩/解压 CSS 代码
正则表达式在线测试
正则表达式在线测试