按照wiki文档中消息主题章节的介绍，为了自定义消息主题，在server.ini中添加了[topic=*]的配置，并且设置了numPartitions属性和[system]中的一致，但是在发送消息的时候，还是抛出There is no aviable partition for topic meta-test,maybe you don't publish it at first?异常，我是采用Spring配置管理的MetaqTemplate来发送Metaq消息的，求解释怎么解决

"SinkRunner-PollingRunner-DefaultSinkProcessor" prio=10 tid=0x00002aaab0653000 nid=0x4141 runnable [0x0000000044169000] java.lang.Thread.State: RUNNABLE at com.taobao.gecko.core.util.LinkedTransferQueue$Itr.advance(LinkedTransferQueue.java:714) at com.taobao.gecko.core.util.LinkedTransferQueue$Itr.(LinkedTransferQueue.java:691) at com.taobao.gecko.core.util.LinkedTransferQueue.iterator(LinkedTransferQueue.java:673) at com.taobao.gecko.core.core.impl.AbstractSession.close0(AbstractSession.java:350) at com.taobao.gecko.core.nio.impl.AbstractNioSession.close0(AbstractNioSession.java:92) at com.taobao.gecko.core.core.impl.AbstractController.stop(AbstractController.java:506) at com.taobao.gecko.service.impl.BaseRemotingController.stop(BaseRemotingController.java:196) - locked <0x00000007b153ac18> (a com.taobao.gecko.service.impl.DefaultRemotingClient) at com.taobao.metamorphosis.client.RemotingClientWrapper.stop(RemotingClientWrapper.java:339) at com.taobao.metamorphosis.client.MetaMessageSessionFactory.shutdown(MetaMessageSessionFactory.java:304) at org.apache.flume.sink.MetaSink.destroyConnection(MetaSink.java:295) at org.apache.flume.sink.MetaSink.process(MetaSink.java:496) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers: - None 这个函数运行几个星期了。 class Itr implements Iterator { QNode nextNode; // Next node to return next QNode currentNode; // last returned node, for remove() QNode prevNode; // predecessor of last returned node E nextItem; // Cache of next item, once commited to in next

    Itr() {
        this.nextNode = traversalHead();
        advance();
    }

    E advance() {
        this.prevNode = this.currentNode;
        this.currentNode = this.nextNode;
        E x = this.nextItem;

        QNode p = this.nextNode.next;
        for (;;) {
            if (p == null || !p.isData) {
                this.nextNode = null;
                this.nextItem = null;
                return x;
            }
            Object item = p.get();
            if (item != p && item != null) {
                this.nextNode = p;
                this.nextItem = cast(item);
                return x;
            }
            this.prevNode = p;
            p = p.next;
        }
    }

    public boolean hasNext() {
        return this.nextNode != null;
    }

    public E next() {
        if (this.nextNode == null) {
            throw new NoSuchElementException();
        }
        return advance();
    }

    public void remove() {
        QNode p = this.currentNode;
        QNode prev = this.prevNode;
        if (prev == null || p == null) {
            throw new IllegalStateException();
        }
        Object x = p.get();
        if (x != null && x != p && p.compareAndSet(x, p)) {
            clean(prev, p);
        }
    }
}

这个for函数应该死循环了 哪位大侠帮忙查一下

public class LinkedTransferQueue extends AbstractQueue implements BlockingQueue { package com.taobao.gecko.core.util; LinkedTransferQueue 这个类在 com.taobao.gecko.core.util;包里面.

不容易也容易(183167601) 14:57:17 714行 是循环那里？ 小规模(245885697) 14:58:15 对应for语句的 } 觉得一直在那个地方for

        messageConsumer.subscribe(topic_order_status_change, 1024, new MessageListener() {
            @Override
            public void recieveMessages(Message message) throws InterruptedException {
                doDispatch(message);
            }

            @Override
            public Executor getExecutor() {
                return Executors.newCachedThreadPool();
            }
        });
        messageConsumer.completeSubscribe();

messageConsumer是单例的，getExecutor方法会不停被调用不产生线程直到整个程序不能创建新线程，如果getExecutor return null的话不存在该问题。

假设一个极端情况：事物准备提交，调用msgStore.append()把message写入messageStore后，执行某一次AppendCallback.appendComplete时程序异常退出。这时消息已保存到数据文件，但是事物由于没有最终完成。

这种情况下事物在recover方法中回滚，已提交的message数据不会被回滚掉。

有一种情况是product，本身就在监听一个端口获取信息后存入broker 为了不让这部分数据再在product和broker中传输占用网络带宽

是否可以 在重载PartitionSelector的getPartition方法后，选择一个本机brokerid的分区（product肯定有办法知道在本机运行的brokerid），让product通过localhost来连接brokered，而不是通过ip来连接，减少网络带宽占用

在Spring 3.2.3.RELEASE中正常，这个错误在Spring 2.5.6.SEC03出现，其他版本没有尝试

错误信息

Cannot convert value of type [java.lang.String] to required type [com.taobao.metamorphosis.client.extension.spring.MetaqTopic]: no matching editors or conversion strategy found

配置文件如下，和wiki中的基本一样，省略了一部分

    <!--  topics to be subscribed. -->
    <bean id = "testTopic" class="com.taobao.metamorphosis.client.extension.spring.MetaqTopic">
        <!-- consumer group -->
        <property name="group" value="testGroup"/>
        <!--  topic -->
        <property name="topic" value="test"/>
        <!--  max buffer size to fetch messages -->
        <property name="maxBufferSize" value="16384"/>
    </bean>
    <!--  listener container to subscribe topics -->
    <bean id ="listenerContainer" class="com.taobao.metamorphosis.client.extension.spring.MessageListenerContainer">
        <property name="messageSessionFactory" ref="sessionFactory"/>
        <property name="messageBodyConverter" ref="messageBodyConverter"/>
        <property name="subscribers">
            <map>
                <entry key-ref="testTopic" value-ref="messageListener"/>
            </map>
        </property>
    </bean>

错误分析

可能是spring初始化MessageListenerContainer的过程中，设置subscribers属性时，没有找到bean id为testTopic的com.taobao.metamorphosis.client.extension.spring.MetaqTopic对象。

<entry key-ref="testTopic" value-ref="messageListener"/>

不知是不是spring 2.5.6.SEC03对这样的泛型支持的不太好呢？

private Map<MetaqTopic/* topic */, ? extends DefaultMessageListener<?>> subscribers

完整的异常信息

[26/06/13 02:37:24:024 CST] main ERROR container.Main:  [DUBBO] Error creating bean with name 'listenerContainer' defined in file [E:\NingYubin\Java\Projects_SurSen\dubbo-metaq-example\service-provider\target\classes\META-INF\spring\surdoc-demo-provider.xml]: Initialization of bean failed; nested exception is org.springframework.beans.TypeMismatchException: Failed to convert property value of type [java.util.LinkedHashMap] to required type [java.util.Map] for property 'subscribers'; nested exception is java.lang.IllegalArgumentException: Cannot convert value of type [java.lang.String] to required type [com.taobao.metamorphosis.client.extension.spring.MetaqTopic]: no matching editors or conversion strategy found, dubbo version: 2.5.3, current host: 127.0.0.1
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'listenerContainer' defined in file [E:\NingYubin\Java\Projects_SurSen\dubbo-metaq-example\service-provider\target\classes\META-INF\spring\surdoc-demo-provider.xml]: Initialization of bean failed; nested exception is org.springframework.beans.TypeMismatchException: Failed to convert property value of type [java.util.LinkedHashMap] to required type [java.util.Map] for property 'subscribers'; nested exception is java.lang.IllegalArgumentException: Cannot convert value of type [java.lang.String] to required type [com.taobao.metamorphosis.client.extension.spring.MetaqTopic]: no matching editors or conversion strategy found
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:480)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:429)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:93)
    at com.alibaba.dubbo.container.spring.SpringContainer.start(SpringContainer.java:50)
    at com.alibaba.dubbo.container.Main.main(Main.java:80)
    at com.alibaba.dubbo.demo.provider.DemoProvider.main(DemoProvider.java:21)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: org.springframework.beans.TypeMismatchException: Failed to convert property value of type [java.util.LinkedHashMap] to required type [java.util.Map] for property 'subscribers'; nested exception is java.lang.IllegalArgumentException: Cannot convert value of type [java.lang.String] to required type [com.taobao.metamorphosis.client.extension.spring.MetaqTopic]: no matching editors or conversion strategy found
    at org.springframework.beans.BeanWrapperImpl.convertForProperty(BeanWrapperImpl.java:391)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.convertForProperty(AbstractAutowireCapableBeanFactory.java:1289)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1250)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1010)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:472)
    ... 21 more
Caused by: java.lang.IllegalArgumentException: Cannot convert value of type [java.lang.String] to required type [com.taobao.metamorphosis.client.extension.spring.MetaqTopic]: no matching editors or conversion strategy found
    at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:231)
    at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:91)
    at org.springframework.beans.BeanWrapperImpl.getPropertyValue(BeanWrapperImpl.java:590)
    at org.springframework.beans.BeanWrapperImpl.getPropertyValue(BeanWrapperImpl.java:532)
    at org.springframework.beans.BeanWrapperImpl.getPropertyType(BeanWrapperImpl.java:293)
    at org.springframework.beans.PropertyEditorRegistrySupport.findCustomEditor(PropertyEditorRegistrySupport.java:301)
    at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:166)
    at org.springframework.beans.TypeConverterDelegate.convertToTypedMap(TypeConverterDelegate.java:520)
    at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:194)
    at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:138)
    at org.springframework.beans.BeanWrapperImpl.convertForProperty(BeanWrapperImpl.java:386)
    ... 25 more

剩下的工作：

• TopicBrowser单元测试和集成测试 done
• MesageListenerContainer单元测试和集成测试 done
• MetaqTemplate集成测试
• 更新spring example done
• 新增XATransactionTemplate API并更新事务例子 done
• 继续编写java客户端指南文档
• XA事务单元测试 done
• 同步复制改进单元测试 done
• gecko升级，设置最大重连尝试次数，彻底解决并发隐患 done
• 压力测试

Consumer实现中，用到的MessageListener，其中，public Executor getExecutor()，返回处理消息的线程池。比如，我采用Executors.newFixedThreadPool(8)返回一个简单的线程池。可是通过计算消息的平均处理时间，发现性能并没有提高多少？这个地方，是否我理解有误？不能这样实现？

设置 deletePolicy=delete,168 系统运行一段时间后，新加入一个consumer group进行消费 此时新加入的group是从168小时前的那个message store文件开始消费 有什么办法可以在group第一次接入的时候，不从那个168小时前的文件进行消费？，而从当前最新message store的offset开始消费

看到介绍说，meta支持事务 动手配置的时候也看到system.ini部分有事务参数的配置

实际编写java的producer部分，看到有事务支持的api

consumer部分没有相关事务支持或者ack相关的api，是否consumer层面不支持事务或者ack相关的特性？

btwl：consumer消费单条message，有没有超时的概念？

客户端提交消息到broker端的时候，偶尔会收到类似这样的异常信息"Clock moved backwards. Refusing to generate id for xxxxx milliseconds"。我们这边会在每天晚上0点进行服务器时间同步调整，但这些异常是在晚上1点多报的。这之间是一直有消息在发送，如果是0点左右报错那可以解释，主要是到了1点多才报错。这个请确认一下。

Hi. I have implemented a tool to detect library version inconsistencies. Your project have 1 inconsistent library and 9 false consistent libraries.

Take mysql:mysql-connector-java for example, this library is declared as version 5.1.6 in metamorphosis-tools, 5.1.17 in metamorphosis-http-client, 5.1.5 in metamorphosis-client and etc... Such version inconsistencies may cause unnecessary maintenance effort in the long run. For example, if two modules become inter-dependent, library version conflict may happen. It has already become a common issue and hinders development progress. Thus a version harmonization is necessary.

Provided we applied a version harmonization, I calculated the cost it may have to harmonize to all upper versions including an up-to-date one. The cost refers to POM config changes and API invocation changes. Take mysql:mysql-connector-java for example, if we harmonize all the library versions into 5.1.17. The concern is, how much should the project code adapt to the newer library version. We list an effort table to quantify the harmonization cost.

The effort table shows the overall harmonization cost on APIs. It seems your project have no API invokes on this library, which could be safely upgrade to 5.1.17

|Index|Module|NA(NAC)|NDA(NDAC)|NMA(NMAC)| |-|-|-|-|-| |1|metamorphosis-tools|0(0)|0(0)|0(0)| |2|metamorphosis-http-client|0(0)|0(0)|0(0)| |3|metamorphosis-client|0(0)|0(0)|0(0)|

Also we provided another table to show the potential files that may be affected due to library API change, which could help to spot the concerned API usage and rerun the test cases.

As for false consistency, take com.taobao.metamorphosis metamorphosis-client jar for example. The library is declared in version 1.4.6.2 in all modules. However they are declared differently. As components are developed in parallel, if one single library version is updated, which could become inconsistent as mentioned above, may cause above-mentioned inconsistency issues

If you are interested, you can have a more complete and detailed report in the attached PDF file. killme2008 Metamorphosis.pdf

Fudan SELab

情况偶现，症状就是程序不退出，进入死循环，CPU使用率100%，关键的堆栈如下：

"Thread-48" daemon prio=10 tid=0x00007f47a8048000 nid=0x73d2 in Object.wait() [0x00007f4796dec000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1289) - locked <0x000000077858ab88> (a java.lang.Thread) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager.interruptRunners(SimpleFetchManager.java:146) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager.stopFetchRunner(SimpleFetchManager.java:128) at com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer.shutdown(SimpleMessageConsumer.java:162) - locked <0x0000000776bdde10> (a com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer) at com.taobao.metamorphosis.client.MetaMessageSessionFactory.shutdown(MetaMessageSessionFactory.java:336) - locked <0x0000000776bdde58> (a com.taobao.metamorphosis.client.MetaMessageSessionFactory) at com.taobao.metamorphosis.client.MetaMessageSessionFactory$2.run(MetaMessageSessionFactory.java:244)

"fetch-Runner-0" daemon prio=10 tid=0x00007f4898002800 nid=0x6bdd runnable [0x00007f49294c5000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Throwable.fillInStackTrace(Native Method) at java.lang.Throwable.fillInStackTrace(Throwable.java:783) - locked <0x000000073760cf20> (a java.lang.InterruptedException) at java.lang.Throwable.(Throwable.java:250) at java.lang.Exception.(Exception.java:54) at java.lang.InterruptedException.(InterruptedException.java:57) at java.lang.Object.wait(Native Method) at com.taobao.gecko.core.nio.impl.SelectorManager.awaitReady(SelectorManager.java:147) - locked <0x00000007702ecd40> (a com.taobao.gecko.core.nio.impl.SelectorManager) at com.taobao.gecko.core.nio.impl.SelectorManager.registerChannel(SelectorManager.java:122) at com.taobao.gecko.core.extension.GeckoTCPConnectorController.connect(GeckoTCPConnectorController.java:75) at com.taobao.gecko.service.impl.DefaultRemotingClient.connect(DefaultRemotingClient.java:120) at com.taobao.gecko.service.impl.DefaultRemotingClient.connect(DefaultRemotingClient.java:148) - locked <0x0000000776b9dbc8> (a com.taobao.gecko.service.impl.DefaultRemotingClient) at com.taobao.metamorphosis.client.RemotingClientWrapper.connectWithRef(RemotingClientWrapper.java:156) - locked <0x0000000770384cb8> (a java.util.HashSet) at com.taobao.metamorphosis.client.RemotingClientWrapper.connectWithRef(RemotingClientWrapper.java:192) at com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer.fetch(SimpleMessageConsumer.java:330) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.processRequest(SimpleFetchManager.java:227) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.run(SimpleFetchManager.java:215) at java.lang.Thread.run(Thread.java:745)

53693 [storm-test-fetch-Runner-0] ERROR c.t.m.c.c.SimpleFetchManager - Process messages failed,topic=test,partition=1 java.lang.RuntimeException: MetaSpout -- Obtain data fail! at com.heetian.bigdata.storm.spout.MetaSpout$1.recieveMessages(MetaSpout.java:139) ~[metaq-storm-0.0.1-SNAPSHOT.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.processReceiveMessage(SimpleFetchManager.java:415) [metamorphosis-client-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.receiveMessages(SimpleFetchManager.java:362) [metamorphosis-client-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.notifyListener(SimpleFetchManager.java:324) [metamorphosis-client-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.processRequest(SimpleFetchManager.java:232) [metamorphosis-client-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.run(SimpleFetchManager.java:215) [metamorphosis-client-1.4.6.2.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73] 54459 [storm-test-fetch-Runner-0] ERROR c.t.m.c.c.SimpleFetchManager - è·åæ¶æ¯å¤±è´¥,topic=test,partition=1 java.lang.NoClassDefFoundError: Could not initialize class com.taobao.metamorphosis.utils.monitor.MonitorLog at com.taobao.metamorphosis.utils.MetaStatLog.addStatValue2(MetaStatLog.java:336) ~[metamorphosis-commons-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.processReceiveMessage(SimpleFetchManager.java:474) ~[metamorphosis-client-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.receiveMessages(SimpleFetchManager.java:362) ~[metamorphosis-client-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.notifyListener(SimpleFetchManager.java:324) [metamorphosis-client-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.processRequest(SimpleFetchManager.java:232) [metamorphosis-client-1.4.6.2.jar:?] at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.run(SimpleFetchManager.java:215) [metamorphosis-client-1.4.6.2.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]