I would suggest that yes, you should just recommend other songs items drawn from the same cluster as the current song.From the way you've phrased your question, it appears you weren't aware of this, but actually, Weka exposes its own API, containing all of the same classes available internally within the GUI. For classes related to clustering, I'd suggest you take a look at EM, XMeans, and Cobweb, although there are also other clustering algorithms that you could use as well. The clustering classes all have a pretty consistent design, there is usually a buildClusterer() method that you can use to build the cluster, and a clusterInstance() method that you can use to retrieve the cluster ID for a given song in the database. I actually built a small Java-based clustering demo project a few months ago in an attempt to try to improve my skills in both Java and Weka at the same time. Feel free to take a look at the source code if you feel it will help.

Does it have to be K-means? Another possible approach is to transform your data into a network first, then apply graph clustering. I am the author of MCL, an algorithm used quite often in bioinformatics. The implementation linked to should easily scale up to networks with millions of nodes - your example would have 300K nodes, assuming that you have 100K attributes. With this approach, the data will be naturally pruned in the data transformation step - and that step will quite likely become the bottleneck. How do you compute the distance between two vectors? In the applications that I have dealt with I used the Pearson or Spearman correlation, and MCL is shipped with software to efficiently perform this computation on large scale data (it can utilise multiple CPUs and multiple machines).

There is still an issue with the data size, as most clustering algorithms will require you to at least perform all pairwise comparisons at least once. Is your data really stored as a giant matrix? Do you have many zeros in the input? Alternatively, do you have a way of discarding smaller elements? Do you have access to more than one machine in order to distribute these computations?

I'm currently trying to cluster a great amount of data points into a given amount of clusters and I wanted to try MOA's streaming based k-means StreamKM. A very simple example of what I'm trying to do using random data looks as follows:

StreamKM streamKM = new StreamKM();
streamKM.numClustersOption.setValue(5); // default setting
streamKM.widthOption.setValue(100000); // default setting
streamKM.prepareForUse();
for (int i = 0; i < 150000; i++) {
    streamKM.trainOnInstanceImpl(randomInstance(2));
}
Clustering result = streamKM.getClusteringResult();
System.out.println("size = " + result.size());
System.out.println("dimension = " + result.dimension());

The random instances are created as follows:

static DenseInstance randomInstance(int size) {
    DenseInstance instance = new DenseInstance(size);
    for (int idx = 0; idx < size; idx++) {
        instance.setValue(idx; Math.random());
    }
    return instance;
}

However; when running the given code; no clusters seem to be created:

System.out.println("size = " + result.size()); // size = 0
System.out.println("dimension = " + result.dimension()); // NPE

Is there anything else I need to take care of; or do I have a fundamental misunderstanding of the MOA clustering concepts?

I have 5 files text. I merge these files into 1 file. That file contain about 60 sentences. I want to clustering that file to 5 cluster. I am using weka to clustering.

public static void doClustering(String pathSentences; int numberCluster) throws IOException {

    Helper.deleteAllFileInFolder("results");

    //so cum bang so cau trong file / so cau trung binh trong 1 file
    HashMap<Integer; String> sentences = new HashMap<>();
    HashMap<Integer; Integer> clustering = new HashMap<>();
    try {
        StringToWordVector filter = new StringToWordVector();
        SimpleKMeans kmeans = new SimpleKMeans();
        FastVector atts = new FastVector(5);
        atts.addElement(new Attribute("text"; (FastVector) null));
        Instances docs = new Instances("text_files"; atts; 0);
        Scanner sc = new Scanner(new File(pathSentences));
        int count = 0;
        while (sc.hasNextLine()) {
            String content = sc.nextLine();
            double[] newInst = new double[1];
            newInst[0] = (double) docs.attribute(0).addStringValue(content);
            docs.add(new SparseInstance(1.0; newInst));
            sentences.put(sentences.size(); content);
            clustering.put(clustering.size(); -1);
        }
        NGramTokenizer tokenizer = new NGramTokenizer();
        tokenizer.setNGramMinSize(10);
        tokenizer.setNGramMaxSize(10);
        tokenizer.setDelimiters("\\W");
        filter.setTokenizer(tokenizer);
        filter.setInputFormat(docs);
        filter.setLowerCaseTokens(true);
        filter.setWordsToKeep(1);
        Instances filteredData = Filter.useFilter(docs; filter);
        kmeans.setPreserveInstancesOrder(true);
        kmeans.setNumClusters(numberCluster);
        kmeans.buildClusterer(filteredData);
        int[] assignments = kmeans.getAssignments();

        int i = 0;
        for (int clusterNum : assignments) {
            clustering.put(i; clusterNum);
            i++;
        }
        PrintWriter[] pw = new PrintWriter[numberCluster];
        for (int j = 0; j < numberCluster; j++) {
            pw[j] = new PrintWriter(new File("results/result" + j + ".txt"));
        }
        sentences.entrySet().stream().forEach((entry) -> {
            Integer key = entry.getKey();
            String value = entry.getValue();
            Integer cluster = clustering.get(key);
            pw[cluster].println(value);
        });
        for (int j = 0; j < numberCluster; j++) {
            pw[j].close();
        }
    } catch (Exception e) {
        System.out.println("Error K means " + e);
    }
}

When I change the order of the input file; the clustering results also vary. Can you help me fix it. Thanks you so much.

I am using the CobWeb provided by Weka; but there is no function to provide the details of centroids of the clusters. I wonder if it is meaningful to get the centroids? If not; which points should I pick to represent every cluster?

I have 3000 sample and 10 class and i want to make clustering them.Because i need to sub cluster of each class. I will use probability of belonging to a sub cluster and probability of belonging to a class. My teacher said to me that i should hierarchical clustering for finding number of clusters. But i cannot take good results. When i increase number of clusters; samples are collecting in a few clusters. Then i have tried elbow method with K-Means. I have looked within cluster sum of squared errors.

I am trying to use a clustering algorithm that lets me choose the initial seeds; so I decided to try and use Weka's Xmeans through the weka GUI. However; when I install Xmeans using weka's package manager; it remains greyed out in the GUI; and I am unable to start clustering even after loading in one of weka's provided test.arff files. Can anyone point me into the right direction or suggest another program or java library to accomplish such a task?

I'm currently trying to cluster a great amount of data points into a given amount of clusters and I wanted to try MOA's streaming based k-means StreamKM. A very simple example of what I'm trying to do using random data looks as follows:

StreamKM streamKM = new StreamKM();
streamKM.numClustersOption.setValue(5); // default setting
streamKM.widthOption.setValue(100000); // default setting
streamKM.prepareForUse();
for (int i = 0; i < 150000; i++) {
    streamKM.trainOnInstanceImpl(randomInstance(2));
}
Clustering result = streamKM.getClusteringResult();
System.out.println("size = " + result.size());
System.out.println("dimension = " + result.dimension());

The random instances are created as follows:

static DenseInstance randomInstance(int size) {
    DenseInstance instance = new DenseInstance(size);
    for (int idx = 0; idx < size; idx++) {
        instance.setValue(idx; Math.random());
    }
    return instance;
}

However; when running the given code; no clusters seem to be created:

System.out.println("size = " + result.size()); // size = 0
System.out.println("dimension = " + result.dimension()); // NPE

Is there anything else I need to take care of; or do I have a fundamental misunderstanding of the MOA clustering concepts?

I'm new to Weka and I am trying to do hierachical clustering. I have a symmetric distance/proximity matrix like this:

    a      b      c      d 
a   0      0.1    0.3     0.2
b   0.1    0      0.7     0.4
c   0.3    0.7    0       0.9
d   0.2    0.4    0.9     0 

I want to do hierarchical agglomerative clustering with these instances(a;b;c;d;..). I installed Weka3.6.11 but I couldn't find any part for passing this distance/proximity matrix in cluster Tab. can anyone help me? Is there any easy way in other environments for this purpose? Thanks in advance.