How to check and modify pids.max of user

1. find out the cgroup of the user

#echo $$|xargs -I '$' cat '/proc/$/cgroup'

/user.slice/user-1000.slice is the cgroup of the user

2. check the pids.max (TasksMax) setting for the user
(1) method 1:
#cgget -a /user.slice/user-1000.slice

(2) method 2:
#cat /sys/fs/cgroup/pids/user.slice/user-1000.slice/pids*

3. modify the pids.max
# systemctl set-property user-1000.slice TasksMax=512

4. verify (using python script)

import time
import threading

def worker():
    while True:
        time.sleep(1000)
        

threads = []
num_worker_threads = 19000
for i in range(num_worker_threads):
    t = threading.Thread(target=worker)
    try:
        t.start()
        threads.append(t)
        print(i)
    except:
        print("EX")
        time.sleep(2000)

for t in threads:
    t.join()

python script:
https://zhuanlan.zhihu.com/p/29192624

cgroup information:
https://access.redhat.com/documentation/zh-tw/red_hat_enterprise_linux/6/html/resource_management_guide/sec-obtaining_information_about_control_groups

xargs:
https://superuser.com/questions/401614/inserting-string-from-xargs-into-another-string

Access control for the multiple cluster transmission queue

https://www.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.con.doc/q017170_.htm

Choose between three modes of checking when an application puts messages to remote cluster queues. The modes are checking remotely against the cluster queue, checking locally against SYSTEM.CLUSTER.TRANSMIT.QUEUE, or checking against local profiles for the cluster queue, or cluster queue manager.

IBM® WebSphere® MQ gives you the choice of checking locally, or locally and remotely, that a user has permission to put a message to a remote queue. A typical IBM WebSphere MQ application uses local checking only, and relies on the remote queue manager trusting the access checks made on the local queue manager. If remote checking is not used, the message is put to the target queue with the authority of the remote message channel process. To use remote checking you must set the put authority of the receiving channel to context security.

The local checks are made against the queue that the application opens. In distributed queueing, the application usually opens a remote queue definition, and access checks are made against the remote queue definition. If the message is put with a full routing header, the checks are made against the transmission queue. If an application opens a cluster queue that is not on the local queue manager, there is no local object to check. The access control checks are made against the cluster transmission queue, SYSTEM.CLUSTER.TRANSMIT.QUEUE. Even with multiple cluster transmission queues, from Version 7.5, local access control checks for remote cluster queues are made against SYSTEM.CLUSTER.TRANSMIT.QUEUE.

The choice of local or remote checking is a choice between two extremes. Checking remotely is fine-grained. Every user must have an access control profile on every queue manager in the cluster to put to any cluster queue. Checking locally is coarse-grained. Every user needs only one access control profile for the cluster transmission queue on the queue manager they are connected to. With that profile, they can put a message to any cluster queue on any queue manager in any cluster.

Since Version 7.1, administrators have another way to set up access control for cluster queues. You can create a security profile for a cluster queue on any queue manager in the cluster using the setmqaut command. The profile takes affect if you open a remote cluster queue locally, specifying only the queue name. You can also set up a profile for a remote queue manager. If you do so, the queue manager can check the profile of a user that opens a cluster queue by providing a fully qualified name.

The new profiles work only if you change the queue manager stanza, ClusterQueueAccessControl to RQMName. The default is Xmitq. You must create profiles for all the cluster queues existing applications use cluster queues. If you change the stanza to RQMName without creating profiles the applications are likely to fail.

ClusterQueueAccessControl usage and its implications

https://www.ibm.com/developerworks/community/blogs/messaging/entry/ClusterQueueAccessControl_usage_and_its_implications?lang=en

From MQ 7.1 onwards, there is a new option made available for the users who can access cluster queue even though they do not have access to SYSTEM.CLUSTER.TRANSMIT.QUEUE. This is made possible with the introduction of ClusterQueueAccessControl attribute in the Security stanza of the qm.ini file.

The ClusterQueueAccessControl attribute takes two values, Xmitq or RQMName. The default value is Xmitq and the behavior is same as seen in previous versions where the access to remote cluster queues gets resolved to the access of the SYSTEM.CLUSTER.TRANSMIT.QUEUE. By setting the ClusterQueueAccessControl to RQMName, the profiles of remote cluster queues will be resolved to locally defined profiles of named queues or named queue manager.

Consider two Qmgrs QM1 & QM2 are part of cluster CLUS1

Create a queue on QM2 and make it part of cluster CLUS1.

  DEFINE QL(MYQ) CLUSTER(CLUS1)

A user who is part of a group called dev_group connects to Qmgr QM1 and wants to put a message to the cluster queue MYQ. By either creating a local named queue profile and giving the user put access on it or by creating a local named queue manager profile and then giving it the required access, the user will be able to put message to the remote cluster queue even though he does not have access over SYSTEM.CLUSTER.TRANSMIT.QUEUE.

1. By creating queue profile :

setmqaut -m QM1 -t queue -n MYQ -g dev_group +putamqsput MYQ QM1

2. By creating queue manager profile :

setmqaut -m QM1 -t rqmname -n QM2 -g dev_group +putamqsput MYQ QM1 8208 0 QM2

qm.ini:

Security:      ClusterQueueAccessControl=RQMName

disable firewall and selinux in CentOS 7

Linux init configuration for test MQ

1. disable selinux
vi /etc/selinux/config
SELINUX=disabled

=> reboot

2. disable firewall
systemctl stop firewalld.service (one time)
systemctl disable firewalld.service (all time)



https://www.opencli.com/linux/centos-7-disable-firewalld-selinux

How to reformat MQ error message

The content of the MQ error log is as following:

12/08/2018 08:21:42 AM - Process(16535.4) User(root) Program(amqzmur0)
                    Host(love) Installation(Installation1)
                    VRMF(8.0.0.9) QMgr(QM2)
                 
AMQ6287: WebSphere MQ V8.0.0.9 (p800-009-180321.1).

EXPLANATION:
WebSphere MQ system information:
Host Info         :- Linux 2.6.32-696.13.2.el6.x86_64 (MQ Linux (x86-64
platform) 64-bit)
Installation      :- /opt/mqm (Installation1)
Version           :- 8.0.0.9 (p800-009-180321.1)
ACTION:
None.
-------------------------------------------------------------------------------
12/08/2018 08:21:41 AM - Process(30069.1) User(root) Program(runmqchl)
                    Host(love) Installation(Installation1)
                    VRMF(8.0.0.9) QMgr(QM2)
                 
AMQ9213: A communications error for TCP/IP occurred.

EXPLANATION:
An unexpected error occurred in communications.
ACTION:
The return code from the TCP/IP (connect) call was 113 (X'71'). Record these
values and tell the systems administrator.
----- amqccita.c : 1278 -------------------------------------------------------
12/08/2018 08:21:41 AM - Process(30069.1) User(root) Program(runmqchl)
                    Host(love) Installation(Installation1)
                    VRMF(8.0.0.9) QMgr(QM2)
                 
AMQ9999: Channel 'QM2.V9QM2' to host '192.168.136.123(1415)' ended abnormally.

EXPLANATION:
The channel program running under process ID 30069 for channel 'QM2.V9QM2'
ended abnormally. The host name is '192.168.136.123(1415)'; in some cases the
host name cannot be determined and so is shown as '????'.
ACTION:
Look at previous error messages for the channel program in the error logs to
determine the cause of the failure. Note that this message can be excluded
AMQERR01.LOG


However I would like to reformat the error log as following:

12/08/2018 08:21:42 AM AMQ6287: WebSphere MQ V8.0.0.9 (p800-009-180321.1).
12/08/2018 08:21:41 AM AMQ9213: A communications error for TCP/IP occurred.
12/08/2018 08:21:41 AM AMQ9999: Channel 'QM2.V9QM2' to host '192.168.136.123(1415)' ended abnormally.

Only grep is not enough to handle, we must ask help from 'sed"

grep -o -E ".*[0-9]{2}:[0-9]{2}:[0-9]{2}\ [A,P]{,1}[M]{,1}[:space:]{,1}|AMQ[0-9]{4}:.*" AMQERR01.LOG|sed '$!N;s/\n/\ /'

grep:

-o: only print the match string
*** AIX 上的opensoure grep 要用[[:space:]]

sed:

N means read the next line (可用來合併二行)
$!N means read the next line unless it is the last line (最後一行不做)

sed   N和$!N 的理解使用

https://blog.51cto.com/goome/1749171

How to do Cold Restart of Queue Manager ?

How to do Cold Restart of Queue Manager ? (or) How to replace the Corrupted Queue Manager logs ?
on June 09, 2014

The following steps are required to cold start MQ on Linux:
 
Replacing Corrupted logs of a QMGR:


1. Delete all current queue manager logs:                                                                                             

mqm@mqm:~> cd /                                                                                                         
mqm@mqm:~> cd /var/mqm/log/OLDQMGR                                                                 
mqm@mqm:~> rm -r *                                                                                                     
                 
2. Create a DUMMY queue manager: 

(Note:  LOG PROPERTIES MUST MATCH OLDQMGR!!!)

If the corrupted QMGR logs looks like below:

   LogPrimaryFiles=55           -lp
   LogSecondaryFiles=2         -ls
   LogFilePages=16384          -lf
   LogType=LINEAR             -ll

Command to duplicate it:

mqm@mqm:~> crtmqm –lp 55 –ls 2 –lf 16384 -ll DUMMY

This might take little time.                                                           

Note: DON'T START the dummy qmgr once it is done.
                                                                                                                                   
3. Move file (amqhlctl.lfh) and directory (active) under:

/var/mqm/log/DUMMY into /var/mqm/log/OLDQMGR                                                                                                                                                                                                                 
mqm@mqm:~> mv /var/mqm/log/DUMMY/*
       
/var/mqm/log/OLDQMGR                                                 
                                                                                                                                   
4. Restart your QMGR:

Note: make sure all qmgr related processes are down:

ps –ef | grep  <qmgrname >

If you see some process still running Kill it with process id just like below:

kill –9 pid                                                                                                     

mqm@mqm:~> strmqm OLDQMGR

                                                                                                                                   
5. Delete the dummy QMGR:
                                                                                                                                   
       mqm@mqm:~> dltmqm DUMMY

Since the logs been replaced, We lost all information to commit or backout messages but all persistent messages are still in the queues (you don't lose persistent messages).


ref:

https://webspheremqadministrator.blogspot.com/2014/06/how-to-do-cold-restart-of-queue-manager.html

Find out which process is using a port in Linux

1. netstat
# netstat -tuplna|grep port_number
[root@love ~]# netstat -tuplan|grep 1477
tcp        0      0 :::1477                     :::*                        LISTEN      12129/runmqlsr
tcp        0      0 ::ffff:127.0.0.1:1477       ::ffff:127.0.0.1:50240      ESTABLISHED 12710/amqrmppa
tcp        0      0 ::ffff:127.0.0.1:50240      ::ffff:127.0.0.1:1477       ESTABLISHED 23447/java

從上面結果可知:
(1)pid 12129 (runmqlsr)    這個process 在LISTEN 1477這個port
(2)pid 12710 (amqrmppa) 這個process用127.0.0.1的1477 port連往 127.0.0.1的50240 port
(3)pid 23447 (java)           這個process 用127.0.0.1的50240 port連往 127.0.0.1的1477 port


2. fuser
#fuser port_number/tcp -u -v
[root@love ~]# fuser 1477/tcp -u -v
                     USER        PID   ACCESS    COMMAND
1477/tcp:      mqm       12129   F....            (mqm)runmqlsr
                     mqm       12710   F....            (mqm)amqrmppa

3. lsof
#lsof -i :port_number
[root@love ~]# lsof -i :1477
COMMAND  PID  USER FD TYPE DEVICE SIZE/OFF NODE NAME
runmqlsr        12129  mqm   5u  IPv6      699704    0t0       TCP *:ms-sna-server (LISTEN)
amqrmppa     12710  mqm   6u  IPv6     2028669   0t0       TCP localhost:ms-sna-server->localhost:50240 (ESTABLISHED)
java               23447   root  118u  IPv6    2028668   0t0       TCP localhost:50240->localhost:ms-sna-server (ESTABLISHED)

Ref:
Linux Find Out Which Process Is Listening Upon a Port

https://www.cyberciti.biz/faq/what-process-has-open-linux-port/